2024-05-27T00:28:43.665 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T00:28:43.665 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727207 --name teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} -- /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727207/orig.config.yaml 2024-05-27T00:28:43.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T00:28:43.820 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727207 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} email: ceph-qa@ceph.com first_in_suite: false job_id: '7727207' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 7d7cb32d1d130135a98121a564d709025c2154b6 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1555 sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 sleep_before_teardown: 0 subset: 10/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7d7cb32d1d130135a98121a564d709025c2154b6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOxAjSbQ+Q70D1x4IdAvOd7URwzF9tc/F+B7k8SVrYziQ+0KjEAVlLnmR/k4bAlQxTfWgpmeCfg0YtVquYjwuak= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUFmpswUyiXukxvmfI3fcXGU4yd0p2rg8Ej0m2fH5DPC0oVAZIhnUzCnlHBVovG59O6kVopt4FKMQkRGHFQn/8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-26_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T00:28:43.820 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa; will attempt to use it 2024-05-27T00:28:43.821 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa/tasks 2024-05-27T00:28:43.822 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T00:28:43.823 INFO:teuthology.task.internal:Checking packages... 2024-05-27T00:28:43.844 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0ef46a246e1118e5adafa1857b4189e7d780c816' 2024-05-27T00:28:43.844 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T00:28:43.844 INFO:teuthology.packaging:ref: None 2024-05-27T00:28:43.844 INFO:teuthology.packaging:tag: None 2024-05-27T00:28:43.844 INFO:teuthology.packaging:branch: squid 2024-05-27T00:28:43.844 INFO:teuthology.packaging:sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-27T00:28:43.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-27T00:28:44.147 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2859.g7d7cb32d 2024-05-27T00:28:44.149 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T00:28:44.155 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T00:28:44.155 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T00:28:44.165 INFO:teuthology.task.internal:Saving configuration 2024-05-27T00:28:44.182 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T00:28:44.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T00:28:44.217 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727207', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 00:23:17.196125', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOxAjSbQ+Q70D1x4IdAvOd7URwzF9tc/F+B7k8SVrYziQ+0KjEAVlLnmR/k4bAlQxTfWgpmeCfg0YtVquYjwuak='} 2024-05-27T00:28:44.243 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727207', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 00:23:17.194305', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUFmpswUyiXukxvmfI3fcXGU4yd0p2rg8Ej0m2fH5DPC0oVAZIhnUzCnlHBVovG59O6kVopt4FKMQkRGHFQn/8='} 2024-05-27T00:28:44.243 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T00:28:44.249 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-27T00:28:44.249 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-27T00:28:44.249 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T00:28:44.317 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f88eced7be0>, signals=[15]) 2024-05-27T00:28:44.317 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T00:28:44.322 INFO:teuthology.task.internal:Opening connections... 2024-05-27T00:28:44.322 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:28:44.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:28:44.400 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:28:44.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:28:44.473 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T00:28:44.480 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-05-27T00:28:44.497 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-05-27T00:28:44.498 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-05-27T00:28:44.555 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-05-27T00:28:44.555 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T00:28:44.556 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T00:28:44.557 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-05-27T00:28:44.581 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-05-27T00:28:44.599 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-05-27T00:28:44.600 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-05-27T00:28:44.656 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-05-27T00:28:44.656 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-05-27T00:28:44.656 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-05-27T00:28:44.657 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-05-27T00:28:44.657 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T00:28:44.658 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T00:28:44.659 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-05-27T00:28:44.681 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T00:28:44.692 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T00:28:44.698 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T00:28:44.698 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-05-27T00:28:44.701 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-05-27T00:28:44.717 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T00:28:44.723 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T00:28:44.723 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-05-27T00:28:44.758 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-05-27T00:28:44.783 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T00:28:44.837 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T00:28:44.852 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-27T00:28:44.852 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-27T00:28:44.852 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-27T00:28:44.853 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T00:28:44.853 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T00:28:44.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:28:44.869 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-05-27T00:28:44.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:28:44.870 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-05-27T00:28:44.925 INFO:teuthology.orchestra.run.smithi153.stdout:5.14.0-437.el9.x86_64 2024-05-27T00:28:44.926 INFO:teuthology.task.kernel:Running kernel on smithi153: 5.14.0-437.el9.x86_64 2024-05-27T00:28:44.926 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum install -y kernel 2024-05-27T00:28:44.927 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-437.el9.x86_64 2024-05-27T00:28:44.927 INFO:teuthology.task.kernel:Running kernel on smithi111: 5.14.0-437.el9.x86_64 2024-05-27T00:28:44.927 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y kernel 2024-05-27T00:28:47.039 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-27T00:28:47.242 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-27T00:28:50.651 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 19 MB 00:02 2024-05-27T00:28:51.615 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-05-27T00:28:54.935 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-27T00:28:55.690 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-05-27T00:28:56.769 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 17 kB 00:00 2024-05-27T00:28:57.291 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 17 kB 00:00 2024-05-27T00:28:58.431 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-27T00:28:59.286 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-27T00:29:03.629 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-27T00:29:04.500 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-05-27T00:29:05.594 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-27T00:29:05.594 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-27T00:29:05.687 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-05-27T00:29:05.697 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout:Install 4 Packages 2024-05-27T00:29:05.698 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:29:05.699 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 78 M 2024-05-27T00:29:05.699 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 123 M 2024-05-27T00:29:05.699 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-05-27T00:29:05.947 INFO:teuthology.orchestra.run.smithi153.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 2.6 MB/s | 576 kB 00:00 2024-05-27T00:29:06.357 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-27T00:29:06.357 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-27T00:29:06.455 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-05-27T00:29:06.462 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-05-27T00:29:06.462 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2024-05-27T00:29:06.462 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-05-27T00:29:06.462 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-05-27T00:29:06.462 INFO:teuthology.orchestra.run.smithi111.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout:Install 4 Packages 2024-05-27T00:29:06.463 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:29:06.464 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 78 M 2024-05-27T00:29:06.464 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 123 M 2024-05-27T00:29:06.464 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-05-27T00:29:07.077 INFO:teuthology.orchestra.run.smithi111.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 1.3 MB/s | 576 kB 00:00 2024-05-27T00:29:07.214 INFO:teuthology.orchestra.run.smithi153.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-27T00:29:08.030 INFO:teuthology.orchestra.run.smithi153.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 15 MB/s | 34 MB 00:02 2024-05-27T00:29:08.339 INFO:teuthology.orchestra.run.smithi153.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-27T00:29:08.339 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-05-27T00:29:08.339 INFO:teuthology.orchestra.run.smithi153.stdout:Total 30 MB/s | 78 MB 00:02 2024-05-27T00:29:08.702 INFO:teuthology.orchestra.run.smithi111.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-05-27T00:29:09.017 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-05-27T00:29:09.319 INFO:teuthology.orchestra.run.smithi111.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 13 MB/s | 34 MB 00:02 2024-05-27T00:29:09.376 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-05-27T00:29:09.376 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-05-27T00:29:09.861 INFO:teuthology.orchestra.run.smithi111.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 10 MB/s | 28 MB 00:02 2024-05-27T00:29:09.861 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-05-27T00:29:09.861 INFO:teuthology.orchestra.run.smithi111.stdout:Total 23 MB/s | 78 MB 00:03 2024-05-27T00:29:10.312 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-05-27T00:29:10.312 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-05-27T00:29:10.539 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-05-27T00:29:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-05-27T00:29:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-05-27T00:29:11.808 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-05-27T00:29:11.808 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-05-27T00:29:12.714 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-05-27T00:29:13.341 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-27T00:29:13.422 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T00:29:14.196 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-05-27T00:29:14.834 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-27T00:29:14.920 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T00:29:16.114 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T00:29:16.247 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T00:29:17.555 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T00:29:17.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T00:29:20.992 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T00:29:21.034 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:22.441 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T00:29:22.494 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:25.751 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:27.211 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:55.439 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:55.476 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:55.889 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:55.889 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-27T00:29:55.889 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T00:29:55.889 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T00:29:56.538 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:56.577 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:56.707 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:56.707 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:29:56.707 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-05-27T00:29:56.708 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-27T00:29:56.708 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-27T00:29:56.708 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-27T00:29:56.708 INFO:teuthology.orchestra.run.smithi153.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-27T00:29:56.708 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:29:56.708 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-05-27T00:29:56.934 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:56.934 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-27T00:29:56.934 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T00:29:56.935 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T00:29:57.044 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2024-05-27T00:29:57.113 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-27T00:29:57.113 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-27T00:29:57.113 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-27T00:29:57.113 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-27T00:29:57.113 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-27T00:29:57.113 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-27T00:29:57.113 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=None, version=distro) 2024-05-27T00:29:57.114 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2024-05-27T00:29:57.163 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-27T00:29:57.164 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-27T00:29:57.165 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -qi grub2-tools 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Name : grub2-tools 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Epoch : 1 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Version : 2.06 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Release : 68.el9 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: x86_64 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Group : Unspecified 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Size : 8264254 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:License : GPLv3+ 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T00:29:57.195 INFO:teuthology.orchestra.run.smithi153.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:Packager : builder@centos.org 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:Vendor : CentOS 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:Summary : Support tools for GRUB. 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:Description : 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:hardware devices. 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:29:57.196 INFO:teuthology.orchestra.run.smithi153.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T00:29:57.197 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T00:29:57.197 INFO:teuthology.task.kernel:Updating grub on smithi153 to boot 5.14.0-447.el9.x86_64 2024-05-27T00:29:57.197 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T00:29:57.725 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T00:29:57.725 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:29:57.725 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-05-27T00:29:57.725 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-27T00:29:57.725 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-27T00:29:57.725 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-27T00:29:57.726 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-27T00:29:57.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:29:57.726 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-05-27T00:29:57.800 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2024-05-27T00:29:58.099 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-05-27T00:29:58.175 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-27T00:29:58.176 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-27T00:29:58.176 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-27T00:29:58.176 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-27T00:29:58.176 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-27T00:29:58.176 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-27T00:29:58.176 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2024-05-27T00:29:58.176 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-05-27T00:29:58.228 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-27T00:29:58.228 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-27T00:29:58.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-05-27T00:29:58.258 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-05-27T00:29:58.258 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-05-27T00:29:58.258 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-05-27T00:29:58.258 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 68.el9 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8264254 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T00:29:58.259 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:29:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T00:29:58.261 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T00:29:58.261 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 5.14.0-447.el9.x86_64 2024-05-27T00:29:58.261 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T00:29:58.855 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-05-27T00:29:59.213 INFO:teuthology.orchestra.run.smithi153.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T00:29:59.242 INFO:teuthology.orchestra.run.smithi153.stderr:done 2024-05-27T00:29:59.244 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2024-05-27T00:29:59.260 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.LAUYxdaIae 2024-05-27T00:29:59.260 DEBUG:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LAUYxdaIae 2024-05-27T00:29:59.326 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.LAUYxdaIae 2024-05-27T00:29:59.466 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.LAUYxdaIae is 6KB 2024-05-27T00:29:59.515 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.LAUYxdaIae 2024-05-27T00:29:59.530 DEBUG:teuthology.orchestra.run.smithi153:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T00:29:59.595 INFO:teuthology.orchestra.run.smithi153.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T00:29:59.595 INFO:teuthology.orchestra.run.smithi153.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T00:29:59.596 INFO:teuthology.orchestra.run.smithi153.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T00:29:59.596 INFO:teuthology.orchestra.run.smithi153.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-27T00:29:59.597 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-27T00:29:59.744 DEBUG:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2024-05-27T00:30:00.190 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T00:30:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-05-27T00:30:00.218 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-05-27T00:30:00.233 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.PVTgwYyJKY 2024-05-27T00:30:00.234 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PVTgwYyJKY 2024-05-27T00:30:00.304 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.PVTgwYyJKY 2024-05-27T00:30:00.442 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.PVTgwYyJKY is 6KB 2024-05-27T00:30:00.490 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.PVTgwYyJKY 2024-05-27T00:30:00.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T00:30:00.566 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T00:30:00.566 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T00:30:00.566 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T00:30:00.567 INFO:teuthology.orchestra.run.smithi111.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-27T00:30:00.568 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-27T00:30:00.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-05-27T00:30:29.774 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-27T00:30:29.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:30:29.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:30:30.717 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-27T00:30:30.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:30:30.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:30:48.095 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-05-27T00:30:57.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:30:57.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:31:00.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-05-27T00:31:05.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-27T00:31:12.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:31:12.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:31:14.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:31:14.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:31:17.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-27T00:31:29.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:31:29.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:31:30.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-05-27T00:31:45.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:31:45.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:31:46.111 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-05-27T00:31:46.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:31:46.533 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-447.el9.x86_64"... 2024-05-27T00:31:46.533 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-05-27T00:31:46.548 INFO:teuthology.orchestra.run.smithi153.stdout:5.14.0-447.el9.x86_64 2024-05-27T00:31:46.548 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-27T00:31:46.548 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T00:31:46.548 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T00:31:47.549 DEBUG:teuthology.parallel:result is None 2024-05-27T00:31:47.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-05-27T00:32:02.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:32:02.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:32:03.143 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-05-27T00:32:03.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:32:03.556 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-27T00:32:03.556 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-05-27T00:32:03.570 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-447.el9.x86_64 2024-05-27T00:32:03.571 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-27T00:32:03.571 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T00:32:03.571 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T00:32:04.572 DEBUG:teuthology.parallel:result is None 2024-05-27T00:32:04.573 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T00:32:04.582 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T00:32:04.582 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T00:32:04.584 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T00:32:04.605 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T00:32:04.661 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T00:32:04.671 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T00:32:04.671 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T00:32:04.674 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T00:32:04.794 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T00:32:04.801 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T00:32:04.801 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T00:32:04.803 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T00:32:04.886 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T00:32:04.908 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T00:32:04.935 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T00:32:04.938 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T00:32:04.940 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T00:32:04.947 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T00:32:04.947 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T00:32:04.949 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T00:32:04.978 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T00:32:04.988 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T00:32:04.988 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T00:32:05.015 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T00:32:05.035 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T00:32:05.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T00:32:05.203 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T00:32:05.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T00:32:05.285 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:32:05.285 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T00:32:05.350 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T00:32:05.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T00:32:05.445 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T00:32:05.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T00:32:05.573 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:32:05.573 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T00:32:05.640 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-05-27T00:32:05.642 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-05-27T00:32:05.685 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T00:32:05.730 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T00:32:06.166 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T00:32:06.174 INFO:teuthology.task.internal:Starting timer... 2024-05-27T00:32:06.174 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T00:32:06.184 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T00:32:06.192 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T00:32:06.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-05-27T00:32:06.268 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-05-27T00:32:06.270 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-05-27T00:32:06.321 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-05-27T00:32:06.323 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T00:32:06.323 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-05-27T00:32:06.345 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-05-27T00:32:06.345 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-05-27T00:32:06.368 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-05-27T00:32:06.368 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive'} 2024-05-27T00:32:06.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T00:32:06.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:32:06.409 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T00:32:06.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:32:06.436 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T00:32:06.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-05-27T00:32:06.490 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-05-27T00:32:06.522 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T00:32:06.532 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-27T00:32:06.535 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-27T00:32:06.684 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T00:32:06.715 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T00:32:06.716 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi111.front.sepia.ceph.com,smithi153.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T00:36:27.848 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com')] 2024-05-27T00:36:27.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:36:27.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:36:27.930 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-05-27T00:36:28.010 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-05-27T00:36:28.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:36:28.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T00:36:28.085 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-05-27T00:36:28.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-05-27T00:36:28.158 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T00:36:28.169 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T00:36:28.170 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T00:36:28.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T00:36:28.172 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T00:36:28.173 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T00:36:28.201 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T00:36:28.216 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T00:36:28.226 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T00:36:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T00:36:28.241 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-05-27T00:36:28.252 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-05-27T00:36:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-05-27T00:36:28.266 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T00:36:28.275 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-05-27T00:36:28.279 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T00:36:28.290 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T00:36:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T00:36:28.334 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-05-27T00:36:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T00:36:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-05-27T00:36:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.358 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-05-27T00:36:28.361 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T00:36:28.361 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-05-27T00:36:28.362 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.362 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.362 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.362 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T00:36:28.363 INFO:teuthology.run_tasks:Running task install... 2024-05-27T00:36:28.371 DEBUG:teuthology.task.install:project ceph 2024-05-27T00:36:28.371 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}} 2024-05-27T00:36:28.371 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'} 2024-05-27T00:36:28.372 INFO:teuthology.task.install:Using flavor: default 2024-05-27T00:36:28.377 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T00:36:28.377 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T00:36:28.377 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'tag': None, 'wait_for_package': False} 2024-05-27T00:36:28.378 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-27T00:36:28.380 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'tag': None, 'wait_for_package': False} 2024-05-27T00:36:28.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-27T00:36:28.631 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/ 2024-05-27T00:36:28.631 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2857.g0ef46a24 2024-05-27T00:36:28.636 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/ 2024-05-27T00:36:28.636 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2857.g0ef46a24 2024-05-27T00:36:28.830 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T00:36:28.830 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:36:28.830 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T00:36:28.839 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T00:36:28.839 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:36:28.839 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T00:36:28.857 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T00:36:28.858 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0ef46a246e1118e5adafa1857b4189e7d780c816/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T00:36:28.865 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T00:36:28.865 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0ef46a246e1118e5adafa1857b4189e7d780c816/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T00:36:28.924 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T00:36:28.930 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T00:36:29.000 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T00:36:29.005 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T00:36:29.065 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-05-27T00:36:29.066 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-05-27T00:36:29.069 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-05-27T00:36:29.071 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-05-27T00:36:29.308 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-05-27T00:36:29.313 INFO:teuthology.orchestra.run.smithi153.stdout:45 files removed 2024-05-27T00:36:29.335 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T00:36:29.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T00:36:30.022 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-05-27T00:36:30.024 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 200 kB/s | 83 kB 00:00 2024-05-27T00:36:30.432 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-05-27T00:36:30.522 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-05-27T00:36:30.825 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-27T00:36:31.014 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-27T00:36:32.573 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.1 MB 00:01 2024-05-27T00:36:32.600 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.1 MB 00:01 2024-05-27T00:36:34.199 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 33 MB/s | 19 MB 00:00 2024-05-27T00:36:38.433 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-05-27T00:36:39.892 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 17 kB 00:00 2024-05-27T00:36:40.744 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-05-27T00:36:45.994 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-27T00:36:46.707 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 19 MB 00:13 2024-05-27T00:36:48.203 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T00:36:48.204 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T00:36:48.214 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T00:36:48.215 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T00:36:48.216 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T00:36:48.217 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T00:36:48.218 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T00:36:48.221 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T00:36:48.222 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T00:36:48.283 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-05-27T00:36:48.310 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-05-27T00:36:48.310 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 6.6 k 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.4 M 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 890 k 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 145 k 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 1.5 M 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 142 k 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 3.5 M 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 7.4 M 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 49 k 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 10 M 2024-05-27T00:36:48.311 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 48 M 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 264 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 762 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 221 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 27 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 18 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 32 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 746 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 127 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T00:36:48.312 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 162 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 321 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 300 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 99 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 85 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 170 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T00:36:48.313 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.4 M 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-05-27T00:36:48.314 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 21 M 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 32 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 2.4 M 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 249 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 4.7 M 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 17 M 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 16 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 25 k 2024-05-27T00:36:48.315 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 163 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T00:36:48.316 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 501 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.3 M 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T00:36:48.317 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 45 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 135 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T00:36:48.318 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T00:36:48.319 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T00:36:48.320 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T00:36:48.321 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T00:36:48.322 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout:======================================================================================= 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout:Install 148 Packages 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 11 Packages 2024-05-27T00:36:48.323 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:36:48.324 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 225 M 2024-05-27T00:36:48.324 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-05-27T00:36:48.837 INFO:teuthology.orchestra.run.smithi153.stdout:(1/159): ceph-19.0.0-2857.g0ef46a24.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-05-27T00:36:49.045 INFO:teuthology.orchestra.run.smithi153.stdout:(2/159): ceph-fuse-19.0.0-2857.g0ef46a24.el9.x8 4.2 MB/s | 890 kB 00:00 2024-05-27T00:36:49.104 INFO:teuthology.orchestra.run.smithi153.stdout:(3/159): ceph-immutable-object-cache-19.0.0-285 2.4 MB/s | 145 kB 00:00 2024-05-27T00:36:49.379 INFO:teuthology.orchestra.run.smithi153.stdout:(4/159): ceph-mds-19.0.0-2857.g0ef46a24.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-05-27T00:36:49.596 INFO:teuthology.orchestra.run.smithi153.stdout:(5/159): ceph-mgr-19.0.0-2857.g0ef46a24.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-05-27T00:36:49.671 INFO:teuthology.orchestra.run.smithi153.stdout:(6/159): ceph-base-19.0.0-2857.g0ef46a24.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-05-27T00:36:50.155 INFO:teuthology.orchestra.run.smithi153.stdout:(7/159): ceph-mon-19.0.0-2857.g0ef46a24.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-05-27T00:36:51.064 INFO:teuthology.orchestra.run.smithi153.stdout:(8/159): ceph-common-19.0.0-2857.g0ef46a24.el9. 8.8 MB/s | 21 MB 00:02 2024-05-27T00:36:51.125 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.2 MB 00:00 2024-05-27T00:36:51.189 INFO:teuthology.orchestra.run.smithi153.stdout:(9/159): ceph-radosgw-19.0.0-2857.g0ef46a24.el9 9.9 MB/s | 10 MB 00:01 2024-05-27T00:36:51.214 INFO:teuthology.orchestra.run.smithi153.stdout:(10/159): ceph-selinux-19.0.0-2857.g0ef46a24.el 167 kB/s | 25 kB 00:00 2024-05-27T00:36:51.339 INFO:teuthology.orchestra.run.smithi153.stdout:(11/159): cephfs-mirror-19.0.0-2857.g0ef46a24.e 1.7 MB/s | 221 kB 00:00 2024-05-27T00:36:51.456 INFO:teuthology.orchestra.run.smithi153.stdout:(12/159): cephfs-shell-19.0.0-2857.g0ef46a24.el 232 kB/s | 27 kB 00:00 2024-05-27T00:36:51.548 INFO:teuthology.orchestra.run.smithi153.stdout:(13/159): libcephfs-devel-19.0.0-2857.g0ef46a24 345 kB/s | 32 kB 00:00 2024-05-27T00:36:51.749 INFO:teuthology.orchestra.run.smithi153.stdout:(14/159): libcephfs2-19.0.0-2857.g0ef46a24.el9. 3.6 MB/s | 746 kB 00:00 2024-05-27T00:36:51.866 INFO:teuthology.orchestra.run.smithi153.stdout:(15/159): libcephsqlite-19.0.0-2857.g0ef46a24.e 1.4 MB/s | 163 kB 00:00 2024-05-27T00:36:52.024 INFO:teuthology.orchestra.run.smithi153.stdout:(16/159): ceph-osd-19.0.0-2857.g0ef46a24.el9.x8 7.1 MB/s | 17 MB 00:02 2024-05-27T00:36:52.049 INFO:teuthology.orchestra.run.smithi153.stdout:(17/159): librados-devel-19.0.0-2857.g0ef46a24. 690 kB/s | 127 kB 00:00 2024-05-27T00:36:52.125 INFO:teuthology.orchestra.run.smithi153.stdout:(18/159): libradosstriper1-19.0.0-2857.g0ef46a2 4.9 MB/s | 501 kB 00:00 2024-05-27T00:36:52.233 INFO:teuthology.orchestra.run.smithi153.stdout:(19/159): python3-ceph-argparse-19.0.0-2857.g0e 417 kB/s | 45 kB 00:00 2024-05-27T00:36:52.334 INFO:teuthology.orchestra.run.smithi153.stdout:(20/159): python3-ceph-common-19.0.0-2857.g0ef4 1.3 MB/s | 135 kB 00:00 2024-05-27T00:36:52.434 INFO:teuthology.orchestra.run.smithi153.stdout:(21/159): python3-cephfs-19.0.0-2857.g0ef46a24. 1.6 MB/s | 162 kB 00:00 2024-05-27T00:36:52.551 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 17 kB 00:00 2024-05-27T00:36:52.568 INFO:teuthology.orchestra.run.smithi153.stdout:(22/159): librgw2-19.0.0-2857.g0ef46a24.el9.x86 10 MB/s | 5.3 MB 00:00 2024-05-27T00:36:52.593 INFO:teuthology.orchestra.run.smithi153.stdout:(23/159): python3-rados-19.0.0-2857.g0ef46a24.e 2.0 MB/s | 321 kB 00:00 2024-05-27T00:36:52.651 INFO:teuthology.orchestra.run.smithi153.stdout:(24/159): python3-rbd-19.0.0-2857.g0ef46a24.el9 3.5 MB/s | 300 kB 00:00 2024-05-27T00:36:52.668 INFO:teuthology.orchestra.run.smithi153.stdout:(25/159): python3-rgw-19.0.0-2857.g0ef46a24.el9 1.3 MB/s | 99 kB 00:00 2024-05-27T00:36:52.727 INFO:teuthology.orchestra.run.smithi153.stdout:(26/159): rbd-fuse-19.0.0-2857.g0ef46a24.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-27T00:36:52.819 INFO:teuthology.orchestra.run.smithi153.stdout:(27/159): rbd-nbd-19.0.0-2857.g0ef46a24.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-27T00:36:52.903 INFO:teuthology.orchestra.run.smithi153.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2857.g 381 kB/s | 32 kB 00:00 2024-05-27T00:36:52.994 INFO:teuthology.orchestra.run.smithi153.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2857.g0ef46a2 1.5 MB/s | 142 kB 00:00 2024-05-27T00:36:53.061 INFO:teuthology.orchestra.run.smithi153.stdout:(30/159): rbd-mirror-19.0.0-2857.g0ef46a24.el9. 7.9 MB/s | 3.1 MB 00:00 2024-05-27T00:36:53.387 INFO:teuthology.orchestra.run.smithi153.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2857.g0ef46 9.0 MB/s | 3.5 MB 00:00 2024-05-27T00:36:53.479 INFO:teuthology.orchestra.run.smithi153.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2857.g0e 2.7 MB/s | 249 kB 00:00 2024-05-27T00:36:53.579 INFO:teuthology.orchestra.run.smithi153.stdout:(33/159): ceph-mgr-rook-19.0.0-2857.g0ef46a24.e 494 kB/s | 49 kB 00:00 2024-05-27T00:36:53.654 INFO:teuthology.orchestra.run.smithi153.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2857.g0 220 kB/s | 16 kB 00:00 2024-05-27T00:36:53.738 INFO:teuthology.orchestra.run.smithi153.stdout:(35/159): ceph-volume-19.0.0-2857.g0ef46a24.el9 3.1 MB/s | 264 kB 00:00 2024-05-27T00:36:53.913 INFO:teuthology.orchestra.run.smithi153.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-05-27T00:36:53.947 INFO:teuthology.orchestra.run.smithi153.stdout:(37/159): cephadm-19.0.0-2857.g0ef46a24.el9.noa 3.6 MB/s | 762 kB 00:00 2024-05-27T00:36:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:(38/159): cephfs-top-19.0.0-2857.g0ef46a24.el9. 242 kB/s | 18 kB 00:00 2024-05-27T00:36:54.489 INFO:teuthology.orchestra.run.smithi153.stdout:(39/159): ceph-test-19.0.0-2857.g0ef46a24.el9.x 15 MB/s | 48 MB 00:03 2024-05-27T00:36:54.514 INFO:teuthology.orchestra.run.smithi153.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 141 kB/s | 80 kB 00:00 2024-05-27T00:36:54.631 INFO:teuthology.orchestra.run.smithi153.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-05-27T00:36:54.657 INFO:teuthology.orchestra.run.smithi153.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-05-27T00:36:54.707 INFO:teuthology.orchestra.run.smithi153.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-27T00:36:54.732 INFO:teuthology.orchestra.run.smithi153.stdout:(44/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-05-27T00:36:54.757 INFO:teuthology.orchestra.run.smithi153.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-27T00:36:54.783 INFO:teuthology.orchestra.run.smithi153.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-27T00:36:54.841 INFO:teuthology.orchestra.run.smithi153.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-27T00:36:54.844 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 21 MB 00:02 2024-05-27T00:36:54.958 INFO:teuthology.orchestra.run.smithi153.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-27T00:36:54.992 INFO:teuthology.orchestra.run.smithi153.stdout:(49/159): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-05-27T00:36:55.017 INFO:teuthology.orchestra.run.smithi153.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-27T00:36:55.067 INFO:teuthology.orchestra.run.smithi153.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-27T00:36:55.092 INFO:teuthology.orchestra.run.smithi153.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-05-27T00:36:55.118 INFO:teuthology.orchestra.run.smithi153.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-05-27T00:36:55.160 INFO:teuthology.orchestra.run.smithi153.stdout:(54/159): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-05-27T00:36:55.185 INFO:teuthology.orchestra.run.smithi153.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-05-27T00:36:55.210 INFO:teuthology.orchestra.run.smithi153.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-05-27T00:36:55.235 INFO:teuthology.orchestra.run.smithi153.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-27T00:36:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-27T00:36:55.294 INFO:teuthology.orchestra.run.smithi153.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-27T00:36:55.336 INFO:teuthology.orchestra.run.smithi153.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-27T00:36:55.362 INFO:teuthology.orchestra.run.smithi153.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-05-27T00:36:55.387 INFO:teuthology.orchestra.run.smithi153.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-27T00:36:55.412 INFO:teuthology.orchestra.run.smithi153.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-27T00:36:55.438 INFO:teuthology.orchestra.run.smithi153.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-27T00:36:55.463 INFO:teuthology.orchestra.run.smithi153.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-27T00:36:55.521 INFO:teuthology.orchestra.run.smithi153.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.0 MB/s | 3.0 MB 00:00 2024-05-27T00:36:55.614 INFO:teuthology.orchestra.run.smithi153.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-05-27T00:36:55.664 INFO:teuthology.orchestra.run.smithi153.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T00:36:55.731 INFO:teuthology.orchestra.run.smithi153.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-27T00:36:55.756 INFO:teuthology.orchestra.run.smithi153.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-27T00:36:55.798 INFO:teuthology.orchestra.run.smithi153.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-05-27T00:36:55.848 INFO:teuthology.orchestra.run.smithi153.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-05-27T00:36:55.873 INFO:teuthology.orchestra.run.smithi153.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-27T00:36:55.907 INFO:teuthology.orchestra.run.smithi153.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-05-27T00:36:55.983 INFO:teuthology.orchestra.run.smithi153.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-27T00:36:55.999 INFO:teuthology.orchestra.run.smithi153.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-27T00:36:56.025 INFO:teuthology.orchestra.run.smithi153.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-05-27T00:36:56.050 INFO:teuthology.orchestra.run.smithi153.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-27T00:36:56.075 INFO:teuthology.orchestra.run.smithi153.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-27T00:36:56.109 INFO:teuthology.orchestra.run.smithi153.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-27T00:36:56.134 INFO:teuthology.orchestra.run.smithi153.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-05-27T00:36:56.218 INFO:teuthology.orchestra.run.smithi153.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-27T00:36:56.243 INFO:teuthology.orchestra.run.smithi153.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-27T00:36:56.268 INFO:teuthology.orchestra.run.smithi153.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-05-27T00:36:56.294 INFO:teuthology.orchestra.run.smithi153.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-27T00:36:56.319 INFO:teuthology.orchestra.run.smithi153.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-05-27T00:36:56.344 INFO:teuthology.orchestra.run.smithi153.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-27T00:36:56.369 INFO:teuthology.orchestra.run.smithi153.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-27T00:36:56.395 INFO:teuthology.orchestra.run.smithi153.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-27T00:36:56.420 INFO:teuthology.orchestra.run.smithi153.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-27T00:36:56.445 INFO:teuthology.orchestra.run.smithi153.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-27T00:36:56.529 INFO:teuthology.orchestra.run.smithi153.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-27T00:36:56.554 INFO:teuthology.orchestra.run.smithi153.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-05-27T00:36:56.580 INFO:teuthology.orchestra.run.smithi153.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-05-27T00:36:56.605 INFO:teuthology.orchestra.run.smithi153.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-27T00:36:56.630 INFO:teuthology.orchestra.run.smithi153.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-27T00:36:56.655 INFO:teuthology.orchestra.run.smithi153.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-05-27T00:36:56.681 INFO:teuthology.orchestra.run.smithi153.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-27T00:36:56.706 INFO:teuthology.orchestra.run.smithi153.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-27T00:36:56.731 INFO:teuthology.orchestra.run.smithi153.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-27T00:36:56.757 INFO:teuthology.orchestra.run.smithi153.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-27T00:36:56.782 INFO:teuthology.orchestra.run.smithi153.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-27T00:36:56.999 INFO:teuthology.orchestra.run.smithi153.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-27T00:36:57.091 INFO:teuthology.orchestra.run.smithi153.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-05-27T00:36:57.133 INFO:teuthology.orchestra.run.smithi153.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-05-27T00:36:57.158 INFO:teuthology.orchestra.run.smithi153.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-05-27T00:36:57.175 INFO:teuthology.orchestra.run.smithi153.stdout:(107/159): lua-devel-5.4.4-4.el9.x86_64.rpm 53 kB/s | 22 kB 00:00 2024-05-27T00:36:57.242 INFO:teuthology.orchestra.run.smithi153.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-05-27T00:36:57.301 INFO:teuthology.orchestra.run.smithi153.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-05-27T00:36:57.326 INFO:teuthology.orchestra.run.smithi153.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-05-27T00:36:57.351 INFO:teuthology.orchestra.run.smithi153.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-27T00:36:57.385 INFO:teuthology.orchestra.run.smithi153.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-27T00:36:57.418 INFO:teuthology.orchestra.run.smithi153.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-05-27T00:36:57.452 INFO:teuthology.orchestra.run.smithi153.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-05-27T00:36:57.477 INFO:teuthology.orchestra.run.smithi153.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-05-27T00:36:57.502 INFO:teuthology.orchestra.run.smithi153.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-05-27T00:36:57.528 INFO:teuthology.orchestra.run.smithi153.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-27T00:36:57.553 INFO:teuthology.orchestra.run.smithi153.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-05-27T00:36:57.578 INFO:teuthology.orchestra.run.smithi153.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-05-27T00:36:57.612 INFO:teuthology.orchestra.run.smithi153.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-05-27T00:36:57.637 INFO:teuthology.orchestra.run.smithi153.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-27T00:36:57.696 INFO:teuthology.orchestra.run.smithi153.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-05-27T00:36:57.721 INFO:teuthology.orchestra.run.smithi153.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-27T00:36:57.746 INFO:teuthology.orchestra.run.smithi153.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-27T00:36:57.771 INFO:teuthology.orchestra.run.smithi153.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-27T00:36:57.797 INFO:teuthology.orchestra.run.smithi153.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-27T00:36:57.822 INFO:teuthology.orchestra.run.smithi153.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-05-27T00:36:57.848 INFO:teuthology.orchestra.run.smithi153.stdout:(128/159): protobuf-compiler-3.14.0-13.el9.x86_ 810 kB/s | 863 kB 00:01 2024-05-27T00:36:57.873 INFO:teuthology.orchestra.run.smithi153.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-27T00:36:57.898 INFO:teuthology.orchestra.run.smithi153.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-05-27T00:36:57.955 INFO:teuthology.orchestra.run.smithi153.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-05-27T00:36:57.982 INFO:teuthology.orchestra.run.smithi153.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 694 kB/s | 58 kB 00:00 2024-05-27T00:36:58.008 INFO:teuthology.orchestra.run.smithi153.stdout:(133/159): python3-more-itertools-8.12.0-2.el9. 588 kB/s | 79 kB 00:00 2024-05-27T00:36:58.033 INFO:teuthology.orchestra.run.smithi153.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-27T00:36:58.058 INFO:teuthology.orchestra.run.smithi153.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-27T00:36:58.083 INFO:teuthology.orchestra.run.smithi153.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-27T00:36:58.109 INFO:teuthology.orchestra.run.smithi153.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-05-27T00:36:58.134 INFO:teuthology.orchestra.run.smithi153.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-05-27T00:36:58.159 INFO:teuthology.orchestra.run.smithi153.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-05-27T00:36:58.184 INFO:teuthology.orchestra.run.smithi153.stdout:(140/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-27T00:36:58.209 INFO:teuthology.orchestra.run.smithi153.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-05-27T00:36:58.235 INFO:teuthology.orchestra.run.smithi153.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-05-27T00:36:58.260 INFO:teuthology.orchestra.run.smithi153.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-27T00:36:58.285 INFO:teuthology.orchestra.run.smithi153.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-27T00:36:58.738 INFO:teuthology.orchestra.run.smithi153.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-05-27T00:36:58.738 INFO:teuthology.orchestra.run.smithi153.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-05-27T00:36:58.738 INFO:teuthology.orchestra.run.smithi153.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-05-27T00:36:58.738 INFO:teuthology.orchestra.run.smithi153.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-27T00:36:58.738 INFO:teuthology.orchestra.run.smithi153.stdout:(149/159): librados2-19.0.0-2857.g0ef46a24.el9. 12 MB/s | 3.4 MB 00:00 2024-05-27T00:36:58.978 INFO:teuthology.orchestra.run.smithi153.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 4.9 MB/s | 1.8 MB 00:00 2024-05-27T00:36:59.029 INFO:teuthology.orchestra.run.smithi153.stdout:(151/159): NetworkManager-1.47.90-1.el9.x86_64. 3.8 MB/s | 2.3 MB 00:00 2024-05-27T00:36:59.054 INFO:teuthology.orchestra.run.smithi153.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 553 kB/s | 41 kB 00:00 2024-05-27T00:36:59.113 INFO:teuthology.orchestra.run.smithi153.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 629 kB/s | 37 kB 00:00 2024-05-27T00:36:59.138 INFO:teuthology.orchestra.run.smithi153.stdout:(154/159): NetworkManager-tui-1.47.90-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-05-27T00:36:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-05-27T00:36:59.222 INFO:teuthology.orchestra.run.smithi153.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.0 MB/s | 260 kB 00:00 2024-05-27T00:36:59.372 INFO:teuthology.orchestra.run.smithi153.stdout:(157/159): librbd1-19.0.0-2857.g0ef46a24.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-05-27T00:36:59.449 INFO:teuthology.orchestra.run.smithi153.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-05-27T00:36:59.673 INFO:teuthology.orchestra.run.smithi153.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.3 MB/s | 2.4 MB 00:00 2024-05-27T00:36:59.676 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-05-27T00:36:59.676 INFO:teuthology.orchestra.run.smithi153.stdout:Total 20 MB/s | 225 MB 00:11 2024-05-27T00:37:00.028 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-27T00:37:00.902 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-05-27T00:37:01.061 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-05-27T00:37:01.061 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-05-27T00:37:02.204 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T00:37:02.204 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T00:37:02.215 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T00:37:02.216 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T00:37:02.217 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T00:37:02.218 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T00:37:02.219 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T00:37:02.222 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T00:37:02.222 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T00:37:02.285 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-05-27T00:37:02.308 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 6.6 k 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.4 M 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 890 k 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 145 k 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 1.5 M 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 142 k 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 3.5 M 2024-05-27T00:37:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 7.4 M 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 49 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 10 M 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 48 M 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 264 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 762 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 221 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 27 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 18 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 32 k 2024-05-27T00:37:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 746 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 127 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 162 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 321 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 300 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 99 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 85 k 2024-05-27T00:37:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 170 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.4 M 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-27T00:37:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 21 M 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 32 k 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 2.4 M 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 249 k 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 4.7 M 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 17 M 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 16 k 2024-05-27T00:37:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 25 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 163 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T00:37:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T00:37:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 501 k 2024-05-27T00:37:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T00:37:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.3 M 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T00:37:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 45 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 135 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T00:37:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T00:37:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T00:37:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T00:37:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T00:37:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 11 Packages 2024-05-27T00:37:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:02.325 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 225 M 2024-05-27T00:37:02.325 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-05-27T00:37:02.696 INFO:teuthology.orchestra.run.smithi111.stdout:(1/159): ceph-19.0.0-2857.g0ef46a24.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-27T00:37:02.930 INFO:teuthology.orchestra.run.smithi111.stdout:(2/159): ceph-fuse-19.0.0-2857.g0ef46a24.el9.x8 3.7 MB/s | 890 kB 00:00 2024-05-27T00:37:03.004 INFO:teuthology.orchestra.run.smithi111.stdout:(3/159): ceph-immutable-object-cache-19.0.0-285 1.9 MB/s | 145 kB 00:00 2024-05-27T00:37:03.255 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-05-27T00:37:03.255 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-05-27T00:37:03.388 INFO:teuthology.orchestra.run.smithi111.stdout:(4/159): ceph-mds-19.0.0-2857.g0ef46a24.el9.x86 6.2 MB/s | 2.4 MB 00:00 2024-05-27T00:37:03.638 INFO:teuthology.orchestra.run.smithi111.stdout:(5/159): ceph-mgr-19.0.0-2857.g0ef46a24.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-05-27T00:37:04.164 INFO:teuthology.orchestra.run.smithi111.stdout:(6/159): ceph-common-19.0.0-2857.g0ef46a24.el9. 13 MB/s | 21 MB 00:01 2024-05-27T00:37:04.230 INFO:teuthology.orchestra.run.smithi111.stdout:(7/159): ceph-base-19.0.0-2857.g0ef46a24.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-05-27T00:37:04.306 INFO:teuthology.orchestra.run.smithi111.stdout:(8/159): ceph-mon-19.0.0-2857.g0ef46a24.el9.x86 7.0 MB/s | 4.7 MB 00:00 2024-05-27T00:37:04.381 INFO:teuthology.orchestra.run.smithi111.stdout:(9/159): ceph-selinux-19.0.0-2857.g0ef46a24.el9 336 kB/s | 25 kB 00:00 2024-05-27T00:37:05.290 INFO:teuthology.orchestra.run.smithi111.stdout:(10/159): ceph-osd-19.0.0-2857.g0ef46a24.el9.x8 15 MB/s | 17 MB 00:01 2024-05-27T00:37:05.373 INFO:teuthology.orchestra.run.smithi111.stdout:(11/159): cephfs-mirror-19.0.0-2857.g0ef46a24.e 2.6 MB/s | 221 kB 00:00 2024-05-27T00:37:05.482 INFO:teuthology.orchestra.run.smithi111.stdout:(12/159): cephfs-shell-19.0.0-2857.g0ef46a24.el 250 kB/s | 27 kB 00:00 2024-05-27T00:37:05.532 INFO:teuthology.orchestra.run.smithi111.stdout:(13/159): libcephfs-devel-19.0.0-2857.g0ef46a24 636 kB/s | 32 kB 00:00 2024-05-27T00:37:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:(14/159): libcephfs2-19.0.0-2857.g0ef46a24.el9. 7.3 MB/s | 746 kB 00:00 2024-05-27T00:37:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:(15/159): libcephsqlite-19.0.0-2857.g0ef46a24.e 2.7 MB/s | 163 kB 00:00 2024-05-27T00:37:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:(16/159): librados-devel-19.0.0-2857.g0ef46a24. 2.1 MB/s | 127 kB 00:00 2024-05-27T00:37:05.833 INFO:teuthology.orchestra.run.smithi111.stdout:(17/159): libradosstriper1-19.0.0-2857.g0ef46a2 5.9 MB/s | 501 kB 00:00 2024-05-27T00:37:06.226 INFO:teuthology.orchestra.run.smithi111.stdout:(18/159): ceph-radosgw-19.0.0-2857.g0ef46a24.el 5.1 MB/s | 10 MB 00:01 2024-05-27T00:37:06.301 INFO:teuthology.orchestra.run.smithi111.stdout:(19/159): librgw2-19.0.0-2857.g0ef46a24.el9.x86 11 MB/s | 5.3 MB 00:00 2024-05-27T00:37:06.326 INFO:teuthology.orchestra.run.smithi111.stdout:(20/159): python3-ceph-argparse-19.0.0-2857.g0e 451 kB/s | 45 kB 00:00 2024-05-27T00:37:06.351 INFO:teuthology.orchestra.run.smithi111.stdout:(21/159): python3-ceph-common-19.0.0-2857.g0ef4 2.6 MB/s | 135 kB 00:00 2024-05-27T00:37:06.402 INFO:teuthology.orchestra.run.smithi111.stdout:(22/159): python3-cephfs-19.0.0-2857.g0ef46a24. 2.1 MB/s | 162 kB 00:00 2024-05-27T00:37:06.427 INFO:teuthology.orchestra.run.smithi111.stdout:(23/159): python3-rados-19.0.0-2857.g0ef46a24.e 4.2 MB/s | 321 kB 00:00 2024-05-27T00:37:06.477 INFO:teuthology.orchestra.run.smithi111.stdout:(24/159): python3-rbd-19.0.0-2857.g0ef46a24.el9 3.9 MB/s | 300 kB 00:00 2024-05-27T00:37:06.502 INFO:teuthology.orchestra.run.smithi111.stdout:(25/159): python3-rgw-19.0.0-2857.g0ef46a24.el9 1.3 MB/s | 99 kB 00:00 2024-05-27T00:37:06.553 INFO:teuthology.orchestra.run.smithi111.stdout:(26/159): rbd-fuse-19.0.0-2857.g0ef46a24.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-27T00:37:06.670 INFO:teuthology.orchestra.run.smithi111.stdout:(27/159): rbd-nbd-19.0.0-2857.g0ef46a24.el9.x86 1.4 MB/s | 170 kB 00:00 2024-05-27T00:37:06.737 INFO:teuthology.orchestra.run.smithi111.stdout:(28/159): rbd-mirror-19.0.0-2857.g0ef46a24.el9. 13 MB/s | 3.1 MB 00:00 2024-05-27T00:37:06.762 INFO:teuthology.orchestra.run.smithi111.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2857.g 345 kB/s | 32 kB 00:00 2024-05-27T00:37:06.787 INFO:teuthology.orchestra.run.smithi111.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2857.g0ef46a2 2.8 MB/s | 142 kB 00:00 2024-05-27T00:37:07.371 INFO:teuthology.orchestra.run.smithi111.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-05-27T00:37:07.496 INFO:teuthology.orchestra.run.smithi111.stdout:(32/159): ceph-mgr-dashboard-19.0.0-2857.g0ef46 4.8 MB/s | 3.5 MB 00:00 2024-05-27T00:37:07.521 INFO:teuthology.orchestra.run.smithi111.stdout:(33/159): ceph-mgr-modules-core-19.0.0-2857.g0e 1.6 MB/s | 249 kB 00:00 2024-05-27T00:37:07.572 INFO:teuthology.orchestra.run.smithi111.stdout:(34/159): ceph-mgr-rook-19.0.0-2857.g0ef46a24.e 656 kB/s | 49 kB 00:00 2024-05-27T00:37:07.597 INFO:teuthology.orchestra.run.smithi111.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2857.g0 218 kB/s | 16 kB 00:00 2024-05-27T00:37:07.656 INFO:teuthology.orchestra.run.smithi111.stdout:(36/159): ceph-volume-19.0.0-2857.g0ef46a24.el9 3.1 MB/s | 264 kB 00:00 2024-05-27T00:37:07.689 INFO:teuthology.orchestra.run.smithi111.stdout:(37/159): cephadm-19.0.0-2857.g0ef46a24.el9.noa 8.1 MB/s | 762 kB 00:00 2024-05-27T00:37:07.715 INFO:teuthology.orchestra.run.smithi111.stdout:(38/159): cephfs-top-19.0.0-2857.g0ef46a24.el9. 310 kB/s | 18 kB 00:00 2024-05-27T00:37:07.832 INFO:teuthology.orchestra.run.smithi111.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-05-27T00:37:07.915 INFO:teuthology.orchestra.run.smithi111.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-27T00:37:07.999 INFO:teuthology.orchestra.run.smithi111.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-27T00:37:08.049 INFO:teuthology.orchestra.run.smithi111.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-27T00:37:08.508 INFO:teuthology.orchestra.run.smithi111.stdout:(43/159): ceph-test-19.0.0-2857.g0ef46a24.el9.x 12 MB/s | 48 MB 00:04 2024-05-27T00:37:08.533 INFO:teuthology.orchestra.run.smithi111.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 220 kB/s | 106 kB 00:00 2024-05-27T00:37:08.617 INFO:teuthology.orchestra.run.smithi111.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-27T00:37:08.675 INFO:teuthology.orchestra.run.smithi111.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 810 kB/s | 135 kB 00:00 2024-05-27T00:37:08.700 INFO:teuthology.orchestra.run.smithi111.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-27T00:37:08.801 INFO:teuthology.orchestra.run.smithi111.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2024-05-27T00:37:08.918 INFO:teuthology.orchestra.run.smithi111.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-05-27T00:37:08.943 INFO:teuthology.orchestra.run.smithi111.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-05-27T00:37:08.960 INFO:teuthology.orchestra.run.smithi111.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 517 kB/s | 22 kB 00:00 2024-05-27T00:37:08.985 INFO:teuthology.orchestra.run.smithi111.stdout:(52/159): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-05-27T00:37:09.027 INFO:teuthology.orchestra.run.smithi111.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-05-27T00:37:09.069 INFO:teuthology.orchestra.run.smithi111.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-27T00:37:09.095 INFO:teuthology.orchestra.run.smithi111.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-05-27T00:37:09.120 INFO:teuthology.orchestra.run.smithi111.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-05-27T00:37:09.145 INFO:teuthology.orchestra.run.smithi111.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-27T00:37:09.170 INFO:teuthology.orchestra.run.smithi111.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-05-27T00:37:09.204 INFO:teuthology.orchestra.run.smithi111.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-27T00:37:09.246 INFO:teuthology.orchestra.run.smithi111.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 614 kB/s | 25 kB 00:00 2024-05-27T00:37:09.280 INFO:teuthology.orchestra.run.smithi111.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-05-27T00:37:09.305 INFO:teuthology.orchestra.run.smithi111.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-05-27T00:37:09.347 INFO:teuthology.orchestra.run.smithi111.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-27T00:37:09.372 INFO:teuthology.orchestra.run.smithi111.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-05-27T00:37:09.397 INFO:teuthology.orchestra.run.smithi111.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-05-27T00:37:09.507 INFO:teuthology.orchestra.run.smithi111.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-05-27T00:37:09.565 INFO:teuthology.orchestra.run.smithi111.stdout:(67/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-05-27T00:37:09.615 INFO:teuthology.orchestra.run.smithi111.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-27T00:37:09.653 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-05-27T00:37:09.657 INFO:teuthology.orchestra.run.smithi111.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-05-27T00:37:09.741 INFO:teuthology.orchestra.run.smithi111.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-05-27T00:37:09.761 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T00:37:09.800 INFO:teuthology.orchestra.run.smithi111.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-27T00:37:09.800 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T00:37:09.883 INFO:teuthology.orchestra.run.smithi111.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-05-27T00:37:09.909 INFO:teuthology.orchestra.run.smithi111.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-05-27T00:37:09.942 INFO:teuthology.orchestra.run.smithi111.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-05-27T00:37:09.959 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T00:37:09.984 INFO:teuthology.orchestra.run.smithi111.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 813 kB/s | 61 kB 00:00 2024-05-27T00:37:10.011 INFO:teuthology.orchestra.run.smithi111.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 3.5 MB/s | 244 kB 00:00 2024-05-27T00:37:10.043 INFO:teuthology.orchestra.run.smithi111.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-05-27T00:37:10.064 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T00:37:10.068 INFO:teuthology.orchestra.run.smithi111.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 451 kB/s | 26 kB 00:00 2024-05-27T00:37:10.094 INFO:teuthology.orchestra.run.smithi111.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-27T00:37:10.094 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T00:37:10.144 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T00:37:10.169 INFO:teuthology.orchestra.run.smithi111.stdout:(80/159): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-05-27T00:37:10.194 INFO:teuthology.orchestra.run.smithi111.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-05-27T00:37:10.219 INFO:teuthology.orchestra.run.smithi111.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-27T00:37:10.253 INFO:teuthology.orchestra.run.smithi111.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-27T00:37:10.290 INFO:teuthology.orchestra.run.smithi111.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-05-27T00:37:10.321 INFO:teuthology.orchestra.run.smithi111.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-27T00:37:10.346 INFO:teuthology.orchestra.run.smithi111.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-05-27T00:37:10.429 INFO:teuthology.orchestra.run.smithi111.stdout:(87/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-05-27T00:37:10.455 INFO:teuthology.orchestra.run.smithi111.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-05-27T00:37:10.473 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T00:37:10.480 INFO:teuthology.orchestra.run.smithi111.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-27T00:37:10.505 INFO:teuthology.orchestra.run.smithi111.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-27T00:37:10.563 INFO:teuthology.orchestra.run.smithi111.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 647 kB/s | 54 kB 00:00 2024-05-27T00:37:10.589 INFO:teuthology.orchestra.run.smithi111.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-05-27T00:37:10.595 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-27T00:37:10.614 INFO:teuthology.orchestra.run.smithi111.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 820 kB/s | 42 kB 00:00 2024-05-27T00:37:10.640 INFO:teuthology.orchestra.run.smithi111.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 875 kB/s | 44 kB 00:00 2024-05-27T00:37:10.682 INFO:teuthology.orchestra.run.smithi111.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-05-27T00:37:10.707 INFO:teuthology.orchestra.run.smithi111.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-05-27T00:37:10.732 INFO:teuthology.orchestra.run.smithi111.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-27T00:37:10.736 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-27T00:37:10.758 INFO:teuthology.orchestra.run.smithi111.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-27T00:37:10.763 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-27T00:37:10.783 INFO:teuthology.orchestra.run.smithi111.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-27T00:37:10.808 INFO:teuthology.orchestra.run.smithi111.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-27T00:37:10.834 INFO:teuthology.orchestra.run.smithi111.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-27T00:37:10.846 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-27T00:37:10.859 INFO:teuthology.orchestra.run.smithi111.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-27T00:37:10.876 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-27T00:37:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-05-27T00:37:10.934 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T00:37:11.001 INFO:teuthology.orchestra.run.smithi111.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 5.9 MB/s | 863 kB 00:00 2024-05-27T00:37:11.126 INFO:teuthology.orchestra.run.smithi111.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.7 MB/s | 548 kB 00:00 2024-05-27T00:37:11.153 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T00:37:11.169 INFO:teuthology.orchestra.run.smithi111.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 470 kB/s | 19 kB 00:00 2024-05-27T00:37:11.194 INFO:teuthology.orchestra.run.smithi111.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-05-27T00:37:11.199 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T00:37:11.227 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-27T00:37:11.236 INFO:teuthology.orchestra.run.smithi111.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-05-27T00:37:11.277 INFO:teuthology.orchestra.run.smithi111.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-27T00:37:11.331 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-27T00:37:11.344 INFO:teuthology.orchestra.run.smithi111.stdout:(110/159): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-27T00:37:11.353 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-27T00:37:11.361 INFO:teuthology.orchestra.run.smithi111.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-05-27T00:37:11.387 INFO:teuthology.orchestra.run.smithi111.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-05-27T00:37:11.416 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-27T00:37:11.437 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T00:37:11.461 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T00:37:11.554 INFO:teuthology.orchestra.run.smithi111.stdout:(113/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-05-27T00:37:11.588 INFO:teuthology.orchestra.run.smithi111.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-05-27T00:37:11.638 INFO:teuthology.orchestra.run.smithi111.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-27T00:37:11.663 INFO:teuthology.orchestra.run.smithi111.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 654 kB/s | 32 kB 00:00 2024-05-27T00:37:11.665 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T00:37:11.688 INFO:teuthology.orchestra.run.smithi111.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-05-27T00:37:11.713 INFO:teuthology.orchestra.run.smithi111.stdout:(118/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 273 kB/s | 43 kB 00:00 2024-05-27T00:37:11.730 INFO:teuthology.orchestra.run.smithi111.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-27T00:37:11.755 INFO:teuthology.orchestra.run.smithi111.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-27T00:37:11.781 INFO:teuthology.orchestra.run.smithi111.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-27T00:37:11.833 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T00:37:11.840 INFO:teuthology.orchestra.run.smithi111.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-05-27T00:37:11.865 INFO:teuthology.orchestra.run.smithi111.stdout:(123/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-27T00:37:11.890 INFO:teuthology.orchestra.run.smithi111.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-27T00:37:11.915 INFO:teuthology.orchestra.run.smithi111.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-05-27T00:37:11.941 INFO:teuthology.orchestra.run.smithi111.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-27T00:37:11.966 INFO:teuthology.orchestra.run.smithi111.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-05-27T00:37:11.991 INFO:teuthology.orchestra.run.smithi111.stdout:(128/159): python3-grpcio-tools-1.46.7-10.el9.x 688 kB/s | 144 kB 00:00 2024-05-27T00:37:12.017 INFO:teuthology.orchestra.run.smithi111.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-27T00:37:12.020 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T00:37:12.059 INFO:teuthology.orchestra.run.smithi111.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-27T00:37:12.084 INFO:teuthology.orchestra.run.smithi111.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 696 kB/s | 46 kB 00:00 2024-05-27T00:37:12.086 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T00:37:12.109 INFO:teuthology.orchestra.run.smithi111.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 858 kB/s | 79 kB 00:00 2024-05-27T00:37:12.135 INFO:teuthology.orchestra.run.smithi111.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-05-27T00:37:12.160 INFO:teuthology.orchestra.run.smithi111.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-05-27T00:37:12.185 INFO:teuthology.orchestra.run.smithi111.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-27T00:37:12.197 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T00:37:12.211 INFO:teuthology.orchestra.run.smithi111.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-27T00:37:12.236 INFO:teuthology.orchestra.run.smithi111.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-05-27T00:37:12.261 INFO:teuthology.orchestra.run.smithi111.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-05-27T00:37:12.271 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T00:37:12.286 INFO:teuthology.orchestra.run.smithi111.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-27T00:37:12.312 INFO:teuthology.orchestra.run.smithi111.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-05-27T00:37:12.337 INFO:teuthology.orchestra.run.smithi111.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-27T00:37:12.362 INFO:teuthology.orchestra.run.smithi111.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-27T00:37:12.387 INFO:teuthology.orchestra.run.smithi111.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-27T00:37:12.413 INFO:teuthology.orchestra.run.smithi111.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-27T00:37:12.438 INFO:teuthology.orchestra.run.smithi111.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-05-27T00:37:12.463 INFO:teuthology.orchestra.run.smithi111.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-27T00:37:12.484 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T00:37:12.489 INFO:teuthology.orchestra.run.smithi111.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-27T00:37:12.539 INFO:teuthology.orchestra.run.smithi111.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-27T00:37:12.597 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T00:37:12.690 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T00:37:12.727 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-27T00:37:12.748 INFO:teuthology.orchestra.run.smithi111.stdout:(149/159): librados2-19.0.0-2857.g0ef46a24.el9. 12 MB/s | 3.4 MB 00:00 2024-05-27T00:37:12.762 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-27T00:37:12.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-27T00:37:12.869 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-27T00:37:12.913 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-27T00:37:12.946 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-27T00:37:12.973 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-27T00:37:13.014 INFO:teuthology.orchestra.run.smithi111.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 5.2 MB/s | 2.3 MB 00:00 2024-05-27T00:37:13.042 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-27T00:37:13.073 INFO:teuthology.orchestra.run.smithi111.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 715 kB/s | 41 kB 00:00 2024-05-27T00:37:13.096 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-27T00:37:13.126 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-27T00:37:13.150 INFO:teuthology.orchestra.run.smithi111.stdout:(152/159): NetworkManager-libnm-1.47.90-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-05-27T00:37:13.169 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-27T00:37:13.174 INFO:teuthology.orchestra.run.smithi111.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 2.4 MB/s | 248 kB 00:00 2024-05-27T00:37:13.224 INFO:teuthology.orchestra.run.smithi111.stdout:(154/159): librbd1-19.0.0-2857.g0ef46a24.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-05-27T00:37:13.245 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-27T00:37:13.249 INFO:teuthology.orchestra.run.smithi111.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 365 kB/s | 37 kB 00:00 2024-05-27T00:37:13.275 INFO:teuthology.orchestra.run.smithi111.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.0 MB/s | 207 kB 00:00 2024-05-27T00:37:13.280 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 34/170 2024-05-27T00:37:13.324 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 35/170 2024-05-27T00:37:13.360 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 36/170 2024-05-27T00:37:13.430 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-27T00:37:13.450 INFO:teuthology.orchestra.run.smithi111.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.1 MB/s | 260 kB 00:00 2024-05-27T00:37:13.475 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-27T00:37:13.483 INFO:teuthology.orchestra.run.smithi111.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-05-27T00:37:13.549 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-27T00:37:13.550 INFO:teuthology.orchestra.run.smithi111.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.7 MB/s | 2.4 MB 00:00 2024-05-27T00:37:13.557 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-05-27T00:37:13.557 INFO:teuthology.orchestra.run.smithi111.stdout:Total 20 MB/s | 225 MB 00:11 2024-05-27T00:37:13.691 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-27T00:37:13.745 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-27T00:37:14.273 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-27T00:37:14.365 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-27T00:37:14.402 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-27T00:37:14.435 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-27T00:37:14.479 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-27T00:37:14.517 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 47/170 2024-05-27T00:37:14.559 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-27T00:37:14.715 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-27T00:37:14.782 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-27T00:37:14.796 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-05-27T00:37:14.834 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-27T00:37:14.907 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T00:37:14.937 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T00:37:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-05-27T00:37:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-05-27T00:37:14.974 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T00:37:15.021 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T00:37:15.057 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T00:37:15.100 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T00:37:15.143 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T00:37:15.280 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T00:37:15.335 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T00:37:15.377 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T00:37:15.472 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T00:37:15.512 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T00:37:15.680 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T00:37:15.761 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T00:37:15.786 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T00:37:15.882 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T00:37:16.012 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T00:37:16.060 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T00:37:16.096 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T00:37:16.141 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T00:37:16.179 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T00:37:16.210 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T00:37:16.240 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T00:37:16.274 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T00:37:16.307 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T00:37:16.349 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T00:37:16.384 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T00:37:16.469 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T00:37:16.514 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T00:37:16.553 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T00:37:16.600 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T00:37:16.699 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T00:37:16.746 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T00:37:17.126 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-05-27T00:37:17.127 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-05-27T00:37:17.465 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T00:37:17.510 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T00:37:17.544 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T00:37:17.643 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T00:37:17.699 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T00:37:17.744 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T00:37:17.840 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T00:37:17.872 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T00:37:18.049 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T00:37:18.078 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T00:37:18.174 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T00:37:18.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T00:37:18.950 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T00:37:20.206 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T00:37:20.290 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T00:37:20.319 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T00:37:20.351 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T00:37:20.709 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T00:37:20.798 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-27T00:37:20.880 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-27T00:37:20.913 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 104/170 2024-05-27T00:37:20.947 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T00:37:21.417 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T00:37:21.443 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-27T00:37:21.514 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-27T00:37:21.545 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 108/170 2024-05-27T00:37:23.444 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-05-27T00:37:23.538 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T00:37:23.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T00:37:23.640 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T00:37:23.680 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T00:37:23.706 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T00:37:23.760 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T00:37:23.848 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-27T00:37:23.883 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-27T00:37:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T00:37:24.198 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-27T00:37:24.321 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-27T00:37:24.358 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-27T00:37:24.434 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-27T00:37:24.449 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-27T00:37:24.470 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-27T00:37:24.479 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-27T00:37:24.541 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T00:37:24.565 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-27T00:37:24.565 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T00:37:24.566 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T00:37:24.566 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:24.596 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-27T00:37:24.759 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T00:37:24.820 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T00:37:24.847 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-27T00:37:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-27T00:37:24.982 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-27T00:37:25.030 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-27T00:37:25.058 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T00:37:25.082 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T00:37:25.286 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T00:37:25.460 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T00:37:25.572 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T00:37:25.641 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T00:37:25.740 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T00:37:25.834 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T00:37:26.040 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T00:37:26.147 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T00:37:26.243 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T00:37:26.281 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-27T00:37:26.315 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-27T00:37:26.380 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-27T00:37:26.419 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-27T00:37:26.466 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-27T00:37:26.500 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-27T00:37:26.527 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-27T00:37:26.597 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-27T00:37:26.650 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-27T00:37:26.680 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-27T00:37:26.723 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-27T00:37:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-27T00:37:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 34/170 2024-05-27T00:37:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 35/170 2024-05-27T00:37:26.898 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 36/170 2024-05-27T00:37:26.971 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-27T00:37:27.021 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-27T00:37:27.097 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-27T00:37:27.241 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-27T00:37:27.291 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-27T00:37:27.817 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-27T00:37:27.897 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-27T00:37:27.932 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-27T00:37:27.969 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-27T00:37:28.009 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-27T00:37:28.047 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 47/170 2024-05-27T00:37:28.089 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-27T00:37:28.238 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-27T00:37:28.309 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-27T00:37:28.363 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-27T00:37:28.436 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T00:37:28.467 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T00:37:28.504 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T00:37:28.551 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T00:37:28.587 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T00:37:28.630 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T00:37:28.672 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T00:37:28.817 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T00:37:28.866 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T00:37:28.906 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T00:37:29.002 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T00:37:29.182 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T00:37:29.224 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T00:37:29.256 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T00:37:29.281 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T00:37:29.371 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T00:37:29.501 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T00:37:29.546 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T00:37:29.575 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T00:37:29.616 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T00:37:29.656 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T00:37:29.688 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T00:37:29.718 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T00:37:29.752 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T00:37:29.785 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T00:37:29.827 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T00:37:29.862 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T00:37:29.940 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T00:37:29.993 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T00:37:30.032 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T00:37:30.079 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T00:37:30.177 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T00:37:30.225 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T00:37:30.952 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T00:37:30.997 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T00:37:31.031 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T00:37:31.129 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T00:37:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T00:37:31.231 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T00:37:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T00:37:31.359 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T00:37:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T00:37:31.573 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T00:37:31.651 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T00:37:32.301 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T00:37:32.422 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T00:37:33.694 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T00:37:33.770 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T00:37:33.807 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T00:37:33.840 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T00:37:34.151 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T00:37:34.195 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-27T00:37:34.268 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-27T00:37:34.299 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 104/170 2024-05-27T00:37:34.335 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T00:37:34.805 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T00:37:34.839 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-27T00:37:34.910 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-27T00:37:34.949 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 108/170 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-05-27T00:37:36.326 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:36.503 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-27T00:37:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-27T00:37:36.519 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:36.519 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T00:37:36.519 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:37.215 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-27T00:37:37.254 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-27T00:37:37.811 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-27T00:37:37.849 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-27T00:37:37.925 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-27T00:37:37.925 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T00:37:37.925 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T00:37:37.925 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:37.959 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-27T00:37:37.997 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-27T00:37:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-27T00:37:38.024 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:38.024 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T00:37:38.024 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T00:37:38.024 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T00:37:38.024 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:38.248 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-27T00:37:38.272 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-27T00:37:38.272 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:38.273 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T00:37:38.273 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T00:37:38.273 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T00:37:38.273 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:38.715 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-27T00:37:38.738 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-27T00:37:38.738 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:38.739 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T00:37:38.739 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T00:37:38.739 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T00:37:38.739 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:38.837 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T00:37:38.894 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T00:37:38.960 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T00:37:39.144 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T00:37:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T00:37:39.269 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T00:37:39.357 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T00:37:39.506 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T00:37:39.541 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T00:37:39.573 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T00:37:39.601 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T00:37:40.071 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-27T00:37:40.101 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-27T00:37:40.131 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 128/170 2024-05-27T00:37:40.193 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 129/170 2024-05-27T00:37:40.223 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-27T00:37:41.262 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-27T00:37:41.308 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-27T00:37:42.279 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-27T00:37:42.317 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-27T00:37:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-27T00:37:42.594 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 133/170 2024-05-27T00:37:42.619 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-27T00:37:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-27T00:37:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T00:37:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T00:37:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T00:37:42.644 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:42.694 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-27T00:37:42.706 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-27T00:37:42.804 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 136/170 2024-05-27T00:37:42.859 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 137/170 2024-05-27T00:37:43.763 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T00:37:43.789 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-27T00:37:43.812 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-27T00:37:43.812 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:43.812 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T00:37:43.812 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T00:37:43.813 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T00:37:43.813 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:43.857 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-27T00:37:43.880 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-27T00:37:43.880 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:43.880 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T00:37:43.880 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:43.932 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-27T00:37:43.955 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-27T00:37:43.955 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:43.956 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T00:37:43.956 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T00:37:43.956 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T00:37:43.956 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:44.266 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-27T00:37:44.290 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-27T00:37:44.290 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:44.290 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T00:37:44.290 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T00:37:44.290 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T00:37:44.290 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:37:48.837 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 143/170 2024-05-27T00:37:48.878 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 144/170 2024-05-27T00:37:48.919 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 145/170 2024-05-27T00:37:48.960 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T00:37:48.987 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T00:37:49.181 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 148/170 2024-05-27T00:37:49.242 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T00:37:49.281 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 150/170 2024-05-27T00:37:49.315 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T00:37:49.344 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T00:37:49.382 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T00:37:49.751 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-27T00:37:49.751 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-05-27T00:37:49.751 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-05-27T00:37:49.751 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-05-27T00:37:49.752 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-05-27T00:37:49.752 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-05-27T00:37:49.752 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-05-27T00:37:49.752 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-05-27T00:37:49.752 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:49.917 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-27T00:37:49.933 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-27T00:37:49.933 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:49.933 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T00:37:49.933 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:50.007 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T00:37:50.046 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T00:37:50.067 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T00:37:50.909 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 155/170 2024-05-27T00:37:50.984 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T00:37:51.025 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T00:37:51.118 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T00:37:51.154 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T00:37:51.179 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T00:37:51.213 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T00:37:51.239 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T00:37:51.239 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T00:37:51.323 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T00:37:51.356 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T00:37:51.356 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T00:37:51.395 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-27T00:37:51.422 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-27T00:37:51.422 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:51.422 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T00:37:51.422 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T00:37:51.422 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T00:37:51.422 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:51.441 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T00:37:51.466 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T00:37:51.471 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T00:37:51.471 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T00:37:51.647 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-27T00:37:51.651 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T00:37:51.651 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T00:37:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-27T00:37:51.669 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:51.669 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T00:37:51.669 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T00:37:51.669 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T00:37:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:51.761 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T00:37:51.785 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T00:37:51.852 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T00:37:52.113 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-27T00:37:52.136 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-27T00:37:52.136 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:52.136 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T00:37:52.137 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T00:37:52.137 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T00:37:52.137 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:52.235 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T00:37:52.300 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T00:37:52.359 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T00:37:52.550 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T00:37:52.597 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T00:37:52.676 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T00:37:52.772 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T00:37:52.922 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T00:37:52.966 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T00:37:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T00:37:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T00:37:53.436 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-27T00:37:53.464 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-27T00:37:53.495 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 128/170 2024-05-27T00:37:53.558 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 129/170 2024-05-27T00:37:53.587 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-27T00:37:54.614 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-27T00:37:54.655 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-27T00:37:55.618 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-27T00:37:55.639 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-27T00:37:55.743 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-27T00:37:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 133/170 2024-05-27T00:37:55.958 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-27T00:37:55.982 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-27T00:37:55.982 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:55.982 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T00:37:55.982 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T00:37:55.982 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T00:37:55.982 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:56.034 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-27T00:37:56.045 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-27T00:37:56.142 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 136/170 2024-05-27T00:37:56.199 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 137/170 2024-05-27T00:37:57.104 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T00:37:57.131 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-27T00:37:57.155 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-27T00:37:57.156 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:57.156 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T00:37:57.156 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T00:37:57.156 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T00:37:57.156 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:57.201 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-27T00:37:57.225 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-27T00:37:57.225 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:57.225 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T00:37:57.225 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:57.269 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-27T00:37:57.292 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-27T00:37:57.292 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:57.292 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T00:37:57.292 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T00:37:57.292 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T00:37:57.292 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:37:57.386 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 1/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 3/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 4/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 5/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 6/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 7/170 2024-05-27T00:37:57.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 11/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 12/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 13/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 15/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 16/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 17/170 2024-05-27T00:37:57.388 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 18/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 19/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 20/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 21/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 22/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 23/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 24/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 25/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 26/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 27/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 28/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 29/170 2024-05-27T00:37:57.389 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 30/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 31/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 32/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 33/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 34/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 35/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 36/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 37/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 38/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 39/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T00:37:57.390 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T00:37:57.391 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T00:37:57.392 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T00:37:57.393 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T00:37:57.394 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T00:37:57.395 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T00:37:57.396 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T00:37:57.397 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T00:37:57.398 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T00:37:57.399 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 149/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 151/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T00:37:57.400 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T00:37:57.401 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T00:37:57.594 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-27T00:37:57.618 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-27T00:37:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T00:37:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T00:37:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T00:37:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T00:37:57.618 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:00.161 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T00:38:00.161 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:00.161 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-05-27T00:38:00.161 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T00:38:00.162 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:00.164 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.165 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T00:38:00.166 INFO:teuthology.orchestra.run.smithi153.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T00:38:00.167 INFO:teuthology.orchestra.run.smithi153.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T00:38:00.168 INFO:teuthology.orchestra.run.smithi153.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T00:38:00.169 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T00:38:00.170 INFO:teuthology.orchestra.run.smithi153.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T00:38:00.171 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T00:38:00.172 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T00:38:00.173 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T00:38:00.174 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T00:38:00.174 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:00.174 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-05-27T00:38:00.602 DEBUG:teuthology.parallel:result is None 2024-05-27T00:38:02.106 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 143/170 2024-05-27T00:38:02.147 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 144/170 2024-05-27T00:38:02.189 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 145/170 2024-05-27T00:38:02.220 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T00:38:02.248 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T00:38:02.448 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 148/170 2024-05-27T00:38:02.493 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T00:38:02.531 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 150/170 2024-05-27T00:38:02.558 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T00:38:02.596 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T00:38:02.642 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T00:38:03.274 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T00:38:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T00:38:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T00:38:04.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 155/170 2024-05-27T00:38:04.270 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T00:38:04.334 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T00:38:04.426 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T00:38:04.456 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T00:38:04.481 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T00:38:04.506 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T00:38:04.533 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T00:38:04.533 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T00:38:04.600 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T00:38:04.633 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T00:38:04.634 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T00:38:04.727 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T00:38:04.751 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T00:38:04.757 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T00:38:04.757 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T00:38:04.945 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T00:38:04.945 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T00:38:05.030 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T00:38:05.054 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T00:38:05.119 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 1/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 3/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 4/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 5/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 6/170 2024-05-27T00:38:10.771 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 7/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 11/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 12/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 13/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 15/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 16/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 17/170 2024-05-27T00:38:10.772 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 18/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 19/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 20/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 21/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 22/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 23/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 24/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 25/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 26/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 27/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 28/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 29/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 30/170 2024-05-27T00:38:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 31/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 32/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 33/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 34/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 35/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 36/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 37/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 38/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 39/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T00:38:10.774 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T00:38:10.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T00:38:10.776 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T00:38:10.777 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T00:38:10.778 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T00:38:10.779 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T00:38:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T00:38:10.781 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T00:38:10.782 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 149/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 151/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T00:38:10.783 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T00:38:10.784 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T00:38:13.416 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T00:38:13.416 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:13.416 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-05-27T00:38:13.416 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:13.416 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T00:38:13.417 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.418 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.419 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T00:38:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T00:38:13.422 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T00:38:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T00:38:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T00:38:13.425 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.426 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T00:38:13.427 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T00:38:13.428 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T00:38:13.429 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T00:38:13.429 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:13.429 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-05-27T00:38:13.897 DEBUG:teuthology.parallel:result is None 2024-05-27T00:38:13.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-27T00:38:14.046 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T00:38:14.069 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-2857.g0ef46a24.el9 2024-05-27T00:38:14.070 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2857.g0ef46a24.el9 2024-05-27T00:38:14.070 INFO:teuthology.task.install:The correct ceph version 19.0.0-2857.g0ef46a24 is installed. 2024-05-27T00:38:14.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-27T00:38:14.217 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T00:38:14.244 INFO:teuthology.orchestra.run.smithi153.stdout:19.0.0-2857.g0ef46a24.el9 2024-05-27T00:38:14.244 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2857.g0ef46a24.el9 2024-05-27T00:38:14.244 INFO:teuthology.task.install:The correct ceph version 19.0.0-2857.g0ef46a24 is installed. 2024-05-27T00:38:14.246 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T00:38:14.247 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:14.247 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T00:38:14.276 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:14.276 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T00:38:14.316 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T00:38:14.316 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:14.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T00:38:14.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T00:38:14.409 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:14.410 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T00:38:14.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T00:38:14.506 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T00:38:14.506 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:14.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T00:38:14.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T00:38:14.597 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:14.597 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T00:38:14.622 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T00:38:14.688 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T00:38:14.688 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:14.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T00:38:14.713 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T00:38:14.778 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:14.778 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T00:38:14.804 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T00:38:14.871 INFO:teuthology.run_tasks:Running task ceph... 2024-05-27T00:38:14.953 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-27T00:38:14.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2024-05-27T00:38:14.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /var/log/ceph 2024-05-27T00:38:14.982 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-27T00:38:14.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-27T00:38:15.022 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-27T00:38:15.046 INFO:tasks.ceph:Creating extra log directories... 2024-05-27T00:38:15.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-27T00:38:15.086 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-27T00:38:15.117 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-27T00:38:15.118 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-27T00:38:15.118 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-27T00:38:15.118 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:15.118 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-27T00:38:15.164 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-27T00:38:15.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-27T00:38:15.297 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:15.297 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-27T00:38:15.312 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-27T00:38:15.424 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-27T00:38:15.454 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-27T00:38:15.454 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-27T00:38:15.454 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727207', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7727207', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '7d7cb32d1d130135a98121a564d709025c2154b6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1555, 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'sleep_before_teardown': 0, 'subset': '10/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7d7cb32d1d130135a98121a564d709025c2154b6', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi111.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOxAjSbQ+Q70D1x4IdAvOd7URwzF9tc/F+B7k8SVrYziQ+0KjEAVlLnmR/k4bAlQxTfWgpmeCfg0YtVquYjwuak=', 'smithi153.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUFmpswUyiXukxvmfI3fcXGU4yd0p2rg8Ej0m2fH5DPC0oVAZIhnUzCnlHBVovG59O6kVopt4FKMQkRGHFQn/8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e7ae7580c417d6ab27fda98fecadca5d9f85d63e', 'timestamp': '2024-05-26_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-27T00:38:15.455 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-27T00:38:15.457 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-27T00:38:15.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2024-05-27T00:38:15.512 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/run/ceph 2024-05-27T00:38:15.569 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:15.569 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T00:38:15.590 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T00:38:15.590 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-27 00:38:05.463470450 +0000 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-27 00:35:48.484168141 +0000 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-27 00:35:48.484168141 +0000 2024-05-27T00:38:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-05-27 00:35:48.484168141 +0000 2024-05-27T00:38:15.647 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T00:38:15.709 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-27T00:38:15.709 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-27T00:38:15.709 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000265664 s, 1.9 MB/s 2024-05-27T00:38:15.710 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T00:38:15.766 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-27 00:38:05.463470450 +0000 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-27 00:35:48.820161770 +0000 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-27 00:35:48.820161770 +0000 2024-05-27T00:38:15.822 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-05-27 00:35:48.820161770 +0000 2024-05-27T00:38:15.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T00:38:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-27T00:38:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-27T00:38:15.885 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000192904 s, 2.7 MB/s 2024-05-27T00:38:15.886 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T00:38:15.943 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-05-27T00:38:15.998 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T00:38:15.998 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:15.998 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-27T00:38:15.999 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:15.999 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:15.999 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-27 00:38:05.464470432 +0000 2024-05-27T00:38:15.999 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-27 00:35:49.143155645 +0000 2024-05-27T00:38:15.999 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-27 00:35:49.143155645 +0000 2024-05-27T00:38:15.999 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-05-27 00:35:49.143155645 +0000 2024-05-27T00:38:15.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T00:38:16.061 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-27T00:38:16.061 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-27T00:38:16.062 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000192171 s, 2.7 MB/s 2024-05-27T00:38:16.063 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T00:38:16.120 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-05-27 00:38:05.464470432 +0000 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-05-27 00:35:49.481149235 +0000 2024-05-27T00:38:16.175 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-05-27 00:35:49.481149235 +0000 2024-05-27T00:38:16.176 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-05-27 00:35:49.481149235 +0000 2024-05-27T00:38:16.176 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T00:38:16.237 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-05-27T00:38:16.237 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-05-27T00:38:16.237 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000191218 s, 2.7 MB/s 2024-05-27T00:38:16.239 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T00:38:16.295 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:16.295 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:16.295 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T00:38:16.312 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T00:38:16.312 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-05-27T00:38:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T00:38:16.369 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:16.369 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-27T00:38:16.370 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:16.370 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:16.370 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-05-27 00:37:52.221196832 +0000 2024-05-27T00:38:16.370 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-05-27 00:35:48.483370314 +0000 2024-05-27T00:38:16.370 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-05-27 00:35:48.483370314 +0000 2024-05-27T00:38:16.370 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-05-27 00:35:48.483370314 +0000 2024-05-27T00:38:16.370 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T00:38:16.434 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-05-27T00:38:16.435 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-05-27T00:38:16.435 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000280175 s, 1.8 MB/s 2024-05-27T00:38:16.436 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T00:38:16.493 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-05-27 00:37:52.221196832 +0000 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-05-27 00:35:48.827364193 +0000 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-05-27 00:35:48.827364193 +0000 2024-05-27T00:38:16.548 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-05-27 00:35:48.827364193 +0000 2024-05-27T00:38:16.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T00:38:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-05-27T00:38:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-05-27T00:38:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000187417 s, 2.7 MB/s 2024-05-27T00:38:16.612 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T00:38:16.668 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-05-27 00:37:52.222196815 +0000 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-05-27 00:35:49.165358178 +0000 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-05-27 00:35:49.165358178 +0000 2024-05-27T00:38:16.724 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-05-27 00:35:49.165358178 +0000 2024-05-27T00:38:16.725 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T00:38:16.786 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-05-27T00:38:16.786 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-05-27T00:38:16.787 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000184626 s, 2.8 MB/s 2024-05-27T00:38:16.788 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T00:38:16.844 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-05-27T00:38:16.899 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T00:38:16.899 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T00:38:16.899 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-27T00:38:16.899 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T00:38:16.899 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T00:38:16.899 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-05-27 00:37:52.222196815 +0000 2024-05-27T00:38:16.900 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-05-27 00:35:49.495352306 +0000 2024-05-27T00:38:16.900 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-05-27 00:35:49.495352306 +0000 2024-05-27T00:38:16.900 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-05-27 00:35:49.495352306 +0000 2024-05-27T00:38:16.900 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T00:38:16.963 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-05-27T00:38:16.963 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-05-27T00:38:16.963 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000188845 s, 2.7 MB/s 2024-05-27T00:38:16.965 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T00:38:17.021 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:17.021 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-27T00:38:17.021 INFO:tasks.ceph:Generating config... 2024-05-27T00:38:17.023 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-27T00:38:17.023 INFO:tasks.ceph:[client] debug client = 20 2024-05-27T00:38:17.023 INFO:tasks.ceph:[client] debug ms = 1 2024-05-27T00:38:17.023 INFO:tasks.ceph:[client] pid file = 2024-05-27T00:38:17.023 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-27T00:38:17.023 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-27T00:38:17.023 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-27T00:38:17.023 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-27T00:38:17.023 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-27T00:38:17.023 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-27T00:38:17.024 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-27T00:38:17.025 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-27T00:38:17.025 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-27T00:38:17.026 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-27T00:38:17.026 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-27T00:38:17.026 INFO:tasks.ceph:Setting up mon.a... 2024-05-27T00:38:17.026 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-27T00:38:17.084 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.keyring 2024-05-27T00:38:17.087 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-27T00:38:17.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T00:38:17.159 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.111'), ('mon.b', '172.21.15.153'), ('mon.c', '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]')] 2024-05-27T00:38:17.159 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.111,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-27T00:38:17.160 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:17.160 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-27T00:38:17.215 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.111 --add b 172.21.15.153 --addv c '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: generated fsid ef97d1dc-a8f7-4487-922a-ccc176f6927a 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:setting min_mon_release = quincy 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 0 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:fsid ef97d1dc-a8f7-4487-922a-ccc176f6927a 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:last_changed 2024-05-27T00:38:17.301055+0000 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:created 2024-05-27T00:38:17.301055+0000 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:min_mon_release 17 (quincy) 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:election_strategy: 1 2024-05-27T00:38:17.301 INFO:teuthology.orchestra.run.smithi111.stdout:0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-05-27T00:38:17.302 INFO:teuthology.orchestra.run.smithi111.stdout:1: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.b 2024-05-27T00:38:17.302 INFO:teuthology.orchestra.run.smithi111.stdout:2: [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] mon.c 2024-05-27T00:38:17.302 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-27T00:38:17.303 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-27T00:38:17.318 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ef97d1dc-a8f7-4487-922a-ccc176f6927a... 2024-05-27T00:38:17.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-27T00:38:17.361 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.401 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-27T00:38:17.402 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2024-05-27T00:38:17.403 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.111,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on skipped message = False 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = ef97d1dc-a8f7-4487-922a-ccc176f6927a 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 600 2024-05-27T00:38:17.405 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock profile = high_recovery_ops 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug trim objects = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.406 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 20 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 20 2024-05-27T00:38:17.407 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 10 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-27T00:38:17.408 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-27T00:38:17.409 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2024-05-27T00:38:17.410 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: mon down mkfs grace = 300 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: mon op complaint time = 120 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2024-05-27T00:38:17.411 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2024-05-27T00:38:17.412 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2024-05-27T00:38:17.413 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2024-05-27T00:38:17.413 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-27T00:38:17.413 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout: client mount timeout = 600 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = "" 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2024-05-27T00:38:17.414 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout:[mds] 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds = 20 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds balancer = 20 2024-05-27T00:38:17.415 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug frag = True 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug scatterstat = True 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: mds op complaint time = 180 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: mds verify scatter = True 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2024-05-27T00:38:17.416 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2024-05-27T00:38:17.418 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.419 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.420 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.111,172.21.15.153,[v2:172.21.15.153:3301,v1:172.21.15.153:6790]" 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on skipped message = False 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = ef97d1dc-a8f7-4487-922a-ccc176f6927a 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.421 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd mclock profile = high_recovery_ops 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug trim objects = true 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2024-05-27T00:38:17.422 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2024-05-27T00:38:17.423 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 20 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 20 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 10 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2024-05-27T00:38:17.424 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2024-05-27T00:38:17.425 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2024-05-27T00:38:17.426 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: mon down mkfs grace = 300 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2024-05-27T00:38:17.427 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2024-05-27T00:38:17.428 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds balancer = 20 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2024-05-27T00:38:17.429 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2024-05-27T00:38:17.429 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-27T00:38:17.430 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-27T00:38:17.511 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-27T00:38:17.511 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:17.512 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-27T00:38:17.527 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:17.527 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-27T00:38:17.583 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:38:17.583 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:17.583 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-27T00:38:17.583 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T00:38:17.657 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:17.657 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:17.713 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:38:17.713 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:17.713 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-27T00:38:17.713 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T00:38:17.748 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:17.748 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:17.804 INFO:tasks.ceph:Setting up mon nodes... 2024-05-27T00:38:17.805 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-27T00:38:17.805 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-27T00:38:17.855 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-27T00:38:17.858 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-27T00:38:17.912 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-27T00:38:17.915 INFO:tasks.ceph:Setting up mds nodes... 2024-05-27T00:38:17.915 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-27T00:38:17.965 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-27T00:38:17.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-27T00:38:17.992 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-27T00:38:18.083 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-27T00:38:18.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-27T00:38:18.110 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-27T00:38:18.161 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-27T00:38:18.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-27T00:38:18.188 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-27T00:38:18.279 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-27T00:38:18.282 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-27T00:38:18.306 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-27T00:38:18.306 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T00:38:18.347 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-27T00:38:18.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-27T00:38:18.440 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-27T00:38:18.452 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-27T00:38:18.493 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-27T00:38:18.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-27T00:38:18.588 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-27T00:38:18.601 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-27T00:38:18.601 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-27T00:38:18.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-27T00:38:18.626 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:18.626 INFO:tasks.ceph:role: osd.0 2024-05-27T00:38:18.626 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:38:18.626 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:18.698 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:18.699 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:18.699 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:18.699 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:19.181 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-27T00:38:19.183 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-27T00:38:19.183 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-27T00:38:19.312 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-27T00:38:19.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-27T00:38:19.411 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:19.411 INFO:tasks.ceph:role: osd.1 2024-05-27T00:38:19.411 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:38:19.412 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-27T00:38:19.485 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:19.485 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:19.485 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:19.485 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:19.486 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:19.486 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:19.486 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:19.486 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:19.486 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:19.486 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:19.946 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-27T00:38:19.948 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-27T00:38:19.948 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-27T00:38:20.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-27T00:38:20.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-27T00:38:20.132 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:20.132 INFO:tasks.ceph:role: osd.2 2024-05-27T00:38:20.132 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:38:20.132 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:20.681 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-27T00:38:20.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-27T00:38:20.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-27T00:38:20.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-27T00:38:20.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-27T00:38:20.917 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:20.917 INFO:tasks.ceph:role: osd.3 2024-05-27T00:38:20.917 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:38:20.917 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-27T00:38:20.989 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:20.989 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:20.989 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:20.989 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:20.989 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:20.990 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:20.990 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:20.990 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:20.990 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:20.990 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:21.506 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-05-27T00:38:21.508 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-05-27T00:38:21.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-27T00:38:21.665 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-27T00:38:21.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:21.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:21.796+0000 7f6a6e521640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-27T00:38:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:21.796+0000 7f6a6e521640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-27T00:38:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:21.796+0000 7f6a6e521640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T00:38:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:21.796+0000 7f6a6e521640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T00:38:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:21.796+0000 7f6a6e521640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T00:38:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:21.797+0000 7f6a6e521640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-27T00:38:24.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T00:38:24.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:24.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:24.763+0000 7fa3d0c43640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-27T00:38:24.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:24.764+0000 7fa3d0c43640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-27T00:38:24.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:24.764+0000 7fa3d0c43640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T00:38:24.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:24.764+0000 7fa3d0c43640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T00:38:24.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:24.764+0000 7fa3d0c43640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T00:38:24.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:24.764+0000 7fa3d0c43640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-27T00:38:27.609 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T00:38:27.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:27.729+0000 7f462460d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-27T00:38:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:27.729+0000 7f462460d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-27T00:38:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:27.729+0000 7f462460d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T00:38:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:27.729+0000 7f462460d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T00:38:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:27.729+0000 7f462460d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T00:38:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:27.730+0000 7f462460d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-27T00:38:30.577 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T00:38:30.605 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:30.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:30.691+0000 7f5e9cc9d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-27T00:38:30.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:30.691+0000 7f5e9cc9d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-27T00:38:30.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:30.691+0000 7f5e9cc9d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T00:38:30.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:30.691+0000 7f5e9cc9d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T00:38:30.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:30.691+0000 7f5e9cc9d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T00:38:30.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:30.692+0000 7f5e9cc9d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-27T00:38:33.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T00:38:33.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-27T00:38:33.598 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:33.598 INFO:tasks.ceph:role: osd.4 2024-05-27T00:38:33.598 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:38:33.598 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-27T00:38:33.674 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:33.675 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:34.188 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-05-27T00:38:34.190 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-05-27T00:38:34.191 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-27T00:38:34.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-27T00:38:34.346 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-27T00:38:34.412 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:34.413 INFO:tasks.ceph:role: osd.5 2024-05-27T00:38:34.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:38:34.413 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:34.486 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:34.964 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-05-27T00:38:34.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-05-27T00:38:34.966 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-27T00:38:35.050 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-27T00:38:35.079 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-27T00:38:35.144 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:35.144 INFO:tasks.ceph:role: osd.6 2024-05-27T00:38:35.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:38:35.144 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-27T00:38:35.216 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:35.216 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:35.216 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:35.216 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:35.217 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:35.217 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:35.217 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:35.217 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:35.217 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:35.217 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:35.712 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-05-27T00:38:35.714 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-05-27T00:38:35.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-27T00:38:35.850 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-27T00:38:35.879 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-27T00:38:35.944 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T00:38:35.944 INFO:tasks.ceph:role: osd.7 2024-05-27T00:38:35.944 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:38:35.944 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-27T00:38:36.015 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T00:38:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T00:38:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T00:38:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T00:38:36.016 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T00:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2024-05-27T00:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T00:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T00:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T00:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T00:38:36.519 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2024-05-27T00:38:36.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2024-05-27T00:38:36.521 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-27T00:38:36.650 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-27T00:38:36.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:36.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:36.779+0000 7f6c8ab16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-27T00:38:36.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:36.779+0000 7f6c8ab16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-27T00:38:36.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:36.779+0000 7f6c8ab16640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T00:38:36.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:36.780+0000 7f6c8ab16640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T00:38:36.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:36.780+0000 7f6c8ab16640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T00:38:36.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:36.780+0000 7f6c8ab16640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-27T00:38:39.638 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T00:38:39.666 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:39.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:39.752+0000 7fb99cc15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-27T00:38:39.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:39.752+0000 7fb99cc15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-27T00:38:39.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:39.752+0000 7fb99cc15640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T00:38:39.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:39.752+0000 7fb99cc15640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T00:38:39.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:39.752+0000 7fb99cc15640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T00:38:39.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:39.752+0000 7fb99cc15640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-27T00:38:42.605 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T00:38:42.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:42.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:42.720+0000 7fa218501640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-27T00:38:42.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:42.720+0000 7fa218501640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-27T00:38:42.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:42.720+0000 7fa218501640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T00:38:42.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:42.720+0000 7fa218501640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T00:38:42.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:42.720+0000 7fa218501640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T00:38:42.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:42.721+0000 7fa218501640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-27T00:38:45.456 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:38:45.459 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:38:45.587 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-27T00:38:45.656 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:45.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:45.746+0000 7f7a5d812640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-27T00:38:45.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:45.747+0000 7f7a5d812640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-27T00:38:45.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:45.747+0000 7f7a5d812640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T00:38:45.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:45.747+0000 7f7a5d812640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T00:38:45.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:45.747+0000 7f7a5d812640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T00:38:45.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:45.747+0000 7f7a5d812640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-27T00:38:48.599 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-27T00:38:48.626 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-27T00:38:48.627 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:48.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-27T00:38:48.656 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:48.657 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-27T00:38:48.725 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:48.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-27T00:38:48.792 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:48.793 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-27T00:38:48.863 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:48.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-27T00:38:48.931 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:48.931 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-27T00:38:49.000 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:49.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-27T00:38:49.068 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.068 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-27T00:38:49.097 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-27T00:38:49.167 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-27T00:38:49.235 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.235 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-27T00:38:49.303 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-27T00:38:49.370 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.370 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-27T00:38:49.437 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.437 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-27T00:38:49.503 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:49.503 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-27T00:38:49.522 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:49.522 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-27T00:38:49.581 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.581 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-27T00:38:49.598 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:49.598 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-27T00:38:49.654 INFO:tasks.ceph:Adding keys to all mons... 2024-05-27T00:38:49.654 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-27T00:38:49.657 DEBUG:teuthology.orchestra.run.smithi153:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-27T00:38:49.697 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.y] 2024-05-27T00:38:49.698 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD51VNmCvv9MhAA0xbdK4Zbp5x+dUa0ewT9XA== 2024-05-27T00:38:49.698 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2024-05-27T00:38:49.698 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD51VNmIrCSORAAVGHebu/KHoXn4GoUa82ycQ== 2024-05-27T00:38:49.698 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2024-05-27T00:38:49.698 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNm57f1BBAAB+W3q8KIOIrG8mA9NSa8pg== 2024-05-27T00:38:49.698 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD91VNm5ZWHLxAA6mGY/B0YPVqLltiybJPeGQ== 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAA1lNmGcaULRAAUiSBUVnQ5ceztu1ynUW/hg== 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAD1lNmBPWFKxAAgSMks9rTqSJjGBPXeixN6A== 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAG1lNm5zdBKRAAxZOTN3UsAoGjOM/SmjyTCw== 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD51VNmU35lNhAA0xN2ysq6ZQy8GctphYmt+g== 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2024-05-27T00:38:49.699 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmCY2lCRAAnMGZi3U691ENXRTH5ZYfsw== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmZ6KqEBAA/W6j7dzA/RwaLjEYDywh4g== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAM1lNmi6SGLhAAev9uBjq1OO/SOEC7j84yqg== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAP1lNmDsrgLBAAvaAx28KGq+5CFEIu9l3lKg== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAS1lNm9FD9KhAAtaruv7gTdRm88QVHPf4CgA== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAV1lNm+VWSLBAAVrsIn004oApYgoF435tgnA== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmQ9y6FBAAIeW3Ce0vjSxE/AG+cOZaUQ== 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout:[client.2] 2024-05-27T00:38:49.700 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmWrk/GhAAC1qM4hIqjC5wLl1I9jdFXQ== 2024-05-27T00:38:49.701 INFO:teuthology.orchestra.run.smithi111.stdout:[client.1] 2024-05-27T00:38:49.701 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmdHZzHRAAXxsw24GAFGbU8daQGuE/dg== 2024-05-27T00:38:49.701 INFO:teuthology.orchestra.run.smithi111.stdout:[client.3] 2024-05-27T00:38:49.701 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmyPcbIxAAYh4qyXALCj9OlT/EmXla8g== 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.y] 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD51VNmCvv9MhAA0xbdK4Zbp5x+dUa0ewT9XA== 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD51VNmIrCSORAAVGHebu/KHoXn4GoUa82ycQ== 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNm57f1BBAAB+W3q8KIOIrG8mA9NSa8pg== 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.0] 2024-05-27T00:38:49.718 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD91VNm5ZWHLxAA6mGY/B0YPVqLltiybJPeGQ== 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.1] 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAA1lNmGcaULRAAUiSBUVnQ5ceztu1ynUW/hg== 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.2] 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAD1lNmBPWFKxAAgSMks9rTqSJjGBPXeixN6A== 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.3] 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAG1lNm5zdBKRAAxZOTN3UsAoGjOM/SmjyTCw== 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.x] 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD51VNmU35lNhAA0xN2ysq6ZQy8GctphYmt+g== 2024-05-27T00:38:49.719 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNmCY2lCRAAnMGZi3U691ENXRTH5ZYfsw== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNmZ6KqEBAA/W6j7dzA/RwaLjEYDywh4g== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.4] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAM1lNmi6SGLhAAev9uBjq1OO/SOEC7j84yqg== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.5] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAP1lNmDsrgLBAAvaAx28KGq+5CFEIu9l3lKg== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.6] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAS1lNm9FD9KhAAtaruv7gTdRm88QVHPf4CgA== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.7] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAV1lNm+VWSLBAAVrsIn004oApYgoF435tgnA== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNmQ9y6FBAAIeW3Ce0vjSxE/AG+cOZaUQ== 2024-05-27T00:38:49.720 INFO:teuthology.orchestra.run.smithi153.stdout:[client.2] 2024-05-27T00:38:49.721 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNmWrk/GhAAC1qM4hIqjC5wLl1I9jdFXQ== 2024-05-27T00:38:49.721 INFO:teuthology.orchestra.run.smithi153.stdout:[client.1] 2024-05-27T00:38:49.721 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNmdHZzHRAAXxsw24GAFGbU8daQGuE/dg== 2024-05-27T00:38:49.721 INFO:teuthology.orchestra.run.smithi153.stdout:[client.3] 2024-05-27T00:38:49.721 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQD61VNmyPcbIxAAYh4qyXALCj9OlT/EmXla8g== 2024-05-27T00:38:49.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T00:38:49.746 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T00:38:49.813 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:49.817 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:49.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:49.873 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:49.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:49.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:49.980 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:49.983 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.034 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.038 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.091 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T00:38:50.144 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T00:38:50.195 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:50.197 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:50.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:50.254 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T00:38:50.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.363 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.413 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.416 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.470 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T00:38:50.521 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.577 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.579 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.635 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.691 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T00:38:50.744 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-27T00:38:50.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-27T00:38:50.769 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T00:38:50.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-27T00:38:50.888 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-27T00:38:50.913 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T00:38:51.010 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-27T00:38:51.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-27T00:38:51.100 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T00:38:51.196 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-27T00:38:51.244 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:51.247 DEBUG:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-27T00:38:51.300 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-27T00:38:51.300 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-27T00:38:51.300 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-27T00:38:51.302 INFO:tasks.ceph.mon.a:Started 2024-05-27T00:38:51.302 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-27T00:38:51.303 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-27T00:38:51.342 INFO:tasks.ceph.mon.b:Started 2024-05-27T00:38:51.342 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-27T00:38:51.342 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-27T00:38:51.344 INFO:tasks.ceph.mon.c:Started 2024-05-27T00:38:51.344 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-27T00:38:51.344 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-27T00:38:51.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-27T00:38:51.347 INFO:tasks.ceph.mgr.y:Started 2024-05-27T00:38:51.347 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-27T00:38:51.347 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-27T00:38:51.349 INFO:tasks.ceph.mgr.x:Started 2024-05-27T00:38:51.349 INFO:tasks.ceph:Setting crush tunables to default 2024-05-27T00:38:51.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd crush tunables default 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.449+0000 7feb6cd90640 1 Processor -- start 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.449+0000 7feb6cd90640 1 -- start start 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6cd90640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6cd90640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6cd90640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680cf2c0 con 0x7feb680cee50 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6cd90640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7feb680ccbf0 con 0x7feb6815c0b0 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6cd90640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feb68070fe0 con 0x7feb680ce0e0 2024-05-27T00:38:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49128/0 (socket says 172.21.15.111:49128) 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 learned_addr learned my addr 172.21.15.111:0/2466549922 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.450+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:51.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:51.651+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T00:38:52.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:52.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:52.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:52.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:52.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:52.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T00:38:52.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T00:38:52.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:52.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:52.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.052+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T00:38:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T00:38:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb6659c640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T00:38:52.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:52.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:52.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:52.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:52.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:52.853+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T00:38:54.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 msgr2=0x7feb680ce910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:54.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb680ce0e0 0x7feb680ce910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:54.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 msgr2=0x7feb680d1920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:54.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.111:3300/0 conn(0x7feb6815c0b0 0x7feb680d1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:54.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 -- 172.21.15.111:0/2466549922 >> v1:172.21.15.153:6789/0 conn(0x7feb680cee50 legacy=0x7feb680cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:54.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 -- 172.21.15.111:0/2466549922 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680b4e40 con 0x7feb48003710 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb64d99640 1 -- 172.21.15.111:0/2466549922 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7feb680ccbf0 con 0x7feb48007380 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.450+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:54.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.651+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:54.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.651+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:54.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.651+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:54.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.651+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:54.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:54.651+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T00:38:55.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.052+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:55.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.052+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:55.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.052+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:55.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.052+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:55.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.052+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T00:38:55.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.854+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:55.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.854+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:55.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.854+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:55.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.854+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:55.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:55.854+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T00:38:56.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.608+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036221063 0 0) 0x7feb680b4e40 con 0x7feb48003710 2024-05-27T00:38:56.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.608+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb54003520 con 0x7feb48003710 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.608+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3910639111 0 0) 0x7feb54003520 con 0x7feb48003710 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.608+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 msgr2=0x7feb48009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.608+0000 7feb65d9b640 1 --2- 172.21.15.111:0/2466549922 >> v2:172.21.15.153:3300/0 conn(0x7feb48007380 0x7feb48009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4800af10 con 0x7feb48003710 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7feb5800f040 con 0x7feb48003710 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 >> v1:172.21.15.111:6789/0 conn(0x7feb48003710 legacy=0x7feb48005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb66d9d640 1 -- 172.21.15.111:0/2466549922 reap_dead start 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540039b0 0x7feb54003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb54004800 0x7feb54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 0x7feb5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:56.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feb680ccbf0 con 0x7feb54004800 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb5400c230 con 0x7feb540039b0 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feb5400c430 con 0x7feb540052e0 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb54004800 0x7feb54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb66d9d640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 0x7feb5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540039b0 0x7feb54003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb66d9d640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 msgr2=0x7feb5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb66d9d640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 msgr2=0x7feb5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb66d9d640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 0x7feb5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb66d9d640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 0x7feb5400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 msgr2=0x7feb5400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 0x7feb5400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540039b0 msgr2=0x7feb54003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:56.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540039b0 0x7feb54003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb5ffff640 1 -- 172.21.15.111:0/2466549922 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54003520 con 0x7feb54004800 2024-05-27T00:38:56.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.609+0000 7feb6659c640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540039b0 0x7feb54003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:38:56.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.610+0000 7feb5ffff640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb54004800 0x7feb54004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feb50006fd0 tx=0x7feb54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896f12811095b4ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:56.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.610+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50032070 con 0x7feb54004800 2024-05-27T00:38:56.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.610+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5002ccd0 con 0x7feb54004800 2024-05-27T00:38:56.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.610+0000 7feb65d9b640 1 -- 172.21.15.111:0/2466549922 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50019940 con 0x7feb54004800 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 -- 172.21.15.111:0/2466549922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb54004800 msgr2=0x7feb54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb54004800 0x7feb54004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feb50006fd0 tx=0x7feb54005830 comp rx=0 tx=0).stop 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 -- 172.21.15.111:0/2466549922 shutdown_connections 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb540052e0 0x7feb5400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb54004800 0x7feb54004c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 --2- 172.21.15.111:0/2466549922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540039b0 0x7feb54003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 -- 172.21.15.111:0/2466549922 >> 172.21.15.111:0/2466549922 conn(0x7feb6806ba70 msgr2=0x7feb680d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 -- 172.21.15.111:0/2466549922 shutdown_connections 2024-05-27T00:38:56.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.611+0000 7feb6cd90640 1 -- 172.21.15.111:0/2466549922 wait complete. 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.613+0000 7feb6cd90640 1 Processor -- start 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.613+0000 7feb6cd90640 1 -- start start 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6cd90640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 0x7feb68151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6cd90640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540052e0 0x7feb6815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6cd90640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 0x7feb680c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6cd90640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feb68070d00 con 0x7feb540039b0 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6cd90640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7feb680ce630 con 0x7feb540052e0 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6cd90640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7feb680cc6d0 con 0x7feb6815a8d0 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 0x7feb68151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 0x7feb68151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49198/0 (socket says 172.21.15.111:49198) 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 -- 172.21.15.111:0/501881588 learned_addr learned my addr 172.21.15.111:0/501881588 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:38:56.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb66d9d640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 0x7feb680c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb5ffff640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540052e0 0x7feb6815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb66d9d640 1 -- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 msgr2=0x7feb680c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb66d9d640 1 -- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 msgr2=0x7feb680c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb66d9d640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 0x7feb680c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb66d9d640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 0x7feb680c7490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 -- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 msgr2=0x7feb680c7490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 0x7feb680c7490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 -- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540052e0 msgr2=0x7feb6815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540052e0 0x7feb6815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.614+0000 7feb6659c640 1 -- 172.21.15.111:0/501881588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb68155090 con 0x7feb540039b0 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb6659c640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 0x7feb68151350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb58015780 tx=0x7feb58015c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58013a60 con 0x7feb540039b0 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb58013c00 con 0x7feb540039b0 2024-05-27T00:38:56.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6815ae10 con 0x7feb540039b0 2024-05-27T00:38:56.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58020480 con 0x7feb540039b0 2024-05-27T00:38:56.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6815b3d0 con 0x7feb540039b0 2024-05-27T00:38:56.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb58020620 con 0x7feb540039b0 2024-05-27T00:38:56.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.615+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb58017da0 con 0x7feb540039b0 2024-05-27T00:38:56.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.616+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb68068680 con 0x7feb540039b0 2024-05-27T00:38:56.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.618+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feb5802e020 con 0x7feb540039b0 2024-05-27T00:38:56.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.684+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7feb6815fa30 con 0x7feb540039b0 2024-05-27T00:38:56.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.686+0000 7feb5effd640 1 -- 172.21.15.111:0/501881588 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7feb58017440 con 0x7feb540039b0 2024-05-27T00:38:56.688 INFO:teuthology.orchestra.run.smithi111.stderr:adjusted tunables profile to default 2024-05-27T00:38:56.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 msgr2=0x7feb68151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:56.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 0x7feb68151350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb58015780 tx=0x7feb58015c40 comp rx=0 tx=0).stop 2024-05-27T00:38:56.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 shutdown_connections 2024-05-27T00:38:56.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7feb6815a8d0 0x7feb680c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb540052e0 0x7feb6815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 --2- 172.21.15.111:0/501881588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb540039b0 0x7feb68151350 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:56.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 >> 172.21.15.111:0/501881588 conn(0x7feb6806ba70 msgr2=0x7feb680c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:56.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 shutdown_connections 2024-05-27T00:38:56.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:56.687+0000 7feb6cd90640 1 -- 172.21.15.111:0/501881588 wait complete. 2024-05-27T00:38:56.699 INFO:tasks.ceph:check_enable_crimson: False 2024-05-27T00:38:56.699 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-27T00:38:56.699 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:56.700 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-27T00:38:56.725 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:56.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-27T00:38:56.728 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:56.728+0000 7fa9b3932040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T00:38:56.791 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:56.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-27T00:38:56.857 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:38:56.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-27T00:38:56.887 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:56.886+0000 7fa9b3932040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T00:38:56.921 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:56.921 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-27T00:38:56.941 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:56.941+0000 7fa9b3932040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T00:38:56.946 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:56.946 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-27T00:38:57.012 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:57.012 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-27T00:38:57.076 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:38:57.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-27T00:38:57.142 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d 0 2024-05-27T00:38:57.204 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:57.202+0000 7fa9b3932040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T00:38:57.252 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:57.251+0000 7fa9b3932040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T00:38:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.281+0000 7f38fae38640 1 Processor -- start 2024-05-27T00:38:57.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.282+0000 7f38fae38640 1 -- start start 2024-05-27T00:38:57.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.282+0000 7f38fae38640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f38f40cf080 0x7f38f4166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.282+0000 7f38fae38640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f40c7700 con 0x7f38f40d19f0 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.282+0000 7f38fae38640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f40c79e0 con 0x7f38f41459c0 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.282+0000 7f38fae38640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40c7cc0 con 0x7f38f40cf080 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f38f41459c0 0x7f38f40c6e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:36680/0 (socket says 172.21.15.153:36680) 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 -- 172.21.15.153:0/1238465736 learned_addr learned my addr 172.21.15.153:0/1238465736 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f93d5640 1 --2- 172.21.15.153:0/1238465736 >> v2:172.21.15.111:3300/0 conn(0x7f38f40cf080 0x7f38f4166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891828617 0 0) 0x7f38f40c79e0 con 0x7f38f41459c0 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38e4003520 con 0x7f38f41459c0 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 323546031 0 0) 0x7f38e4003520 con 0x7f38f41459c0 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 >> v2:172.21.15.111:3300/0 conn(0x7f38f40cf080 msgr2=0x7f38f4166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 --2- 172.21.15.153:0/1238465736 >> v2:172.21.15.111:3300/0 conn(0x7f38f40cf080 0x7f38f4166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 >> v1:172.21.15.111:6789/0 conn(0x7f38f40d19f0 legacy=0x7f38f40ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f40c7fa0 con 0x7f38f41459c0 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f38e8001670 con 0x7f38f41459c0 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 >> v1:172.21.15.153:6789/0 conn(0x7f38f41459c0 legacy=0x7f38f40c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 0x7f38e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38e4004800 0x7f38e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e40051c0 0x7f38e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40c7cc0 con 0x7f38e40051c0 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38e400c160 con 0x7f38e4004800 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f38e400c360 con 0x7f38e40039b0 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 0x7f38e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f3fff640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38e4004800 0x7f38e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f93d5640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e40051c0 0x7f38e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 -- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 msgr2=0x7f38e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 -- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 msgr2=0x7f38e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 0x7f38e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.283+0000 7f38f8bd4640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 0x7f38e4003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f3fff640 1 -- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 msgr2=0x7f38e4003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f3fff640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38e40039b0 0x7f38e4003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f3fff640 1 -- 172.21.15.153:0/1238465736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e40051c0 msgr2=0x7f38e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f3fff640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e40051c0 0x7f38e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f3fff640 1 -- 172.21.15.153:0/1238465736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e4003520 con 0x7f38e4004800 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f93d5640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e40051c0 0x7f38e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f93d5640 1 -- 172.21.15.153:0/1238465736 reap_dead start 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f3fff640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38e4004800 0x7f38e4004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001720 tx=0x7f38e4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0e6235226bde8d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0001370 con 0x7f38e4004800 2024-05-27T00:38:57.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e00050d0 con 0x7f38e4004800 2024-05-27T00:38:57.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.284+0000 7f38f37fe640 1 -- 172.21.15.153:0/1238465736 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e00035f0 con 0x7f38e4004800 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 -- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38e4004800 msgr2=0x7f38e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38e4004800 0x7f38e4004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001720 tx=0x7f38e4005830 comp rx=0 tx=0).stop 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 -- 172.21.15.153:0/1238465736 shutdown_connections 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 --2- 172.21.15.153:0/1238465736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38e4004800 0x7f38e4004c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 -- 172.21.15.153:0/1238465736 >> 172.21.15.153:0/1238465736 conn(0x7f38f40d1f60 msgr2=0x7f38f40d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 -- 172.21.15.153:0/1238465736 shutdown_connections 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.285+0000 7f38fae38640 1 -- 172.21.15.153:0/1238465736 wait complete. 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.286+0000 7f38fae38640 1 Processor -- start 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 -- start start 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e4004800 0x7f38f40b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38f415ac10 0x7f38f415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40ceaf0 con 0x7f38e4004800 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40ce970 con 0x7f38f415ac10 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38fae38640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f38f40cec70 con 0x7f38f40b5140 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38f93d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38f415ac10 0x7f38f415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38f3fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:43982/0 (socket says 172.21.15.153:43982) 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.287+0000 7f38f3fff640 1 -- 172.21.15.153:0/3698361210 learned_addr learned my addr 172.21.15.153:0/3698361210 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f3fff640 1 -- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 msgr2=0x7f38f40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f3fff640 1 -- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 msgr2=0x7f38f40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f3fff640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f3fff640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f8bd4640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e4004800 0x7f38f40b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f93d5640 1 -- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 msgr2=0x7f38f40cd280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f93d5640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f93d5640 1 -- 172.21.15.153:0/3698361210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e4004800 msgr2=0x7f38f40b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f93d5640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e4004800 0x7f38f40b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f93d5640 1 -- 172.21.15.153:0/3698361210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f415b600 con 0x7f38f415ac10 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f93d5640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38f415ac10 0x7f38f415b0c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f38f40b4c00 tx=0x7f38ec00f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec009c10 con 0x7f38f415ac10 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec00fdd0 con 0x7f38f415ac10 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec003bd0 con 0x7f38f415ac10 2024-05-27T00:38:57.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f40c9d70 con 0x7f38f415ac10 2024-05-27T00:38:57.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f40ca330 con 0x7f38f415ac10 2024-05-27T00:38:57.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38ec01f020 con 0x7f38f415ac10 2024-05-27T00:38:57.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.288+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f38ec0218e0 con 0x7f38f415ac10 2024-05-27T00:38:57.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.289+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f4068680 con 0x7f38f415ac10 2024-05-27T00:38:57.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.291+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f38ec026020 con 0x7f38f415ac10 2024-05-27T00:38:57.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.365+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d", "id": 0} v 0) -- 0x7f38f40b3140 con 0x7f38f415ac10 2024-05-27T00:38:57.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.370+0000 7f38f17fa640 1 -- 172.21.15.153:0/3698361210 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f38f415b600 con 0x7f38f415ac10 2024-05-27T00:38:57.371 INFO:teuthology.orchestra.run.smithi153.stdout:0 2024-05-27T00:38:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38f415ac10 msgr2=0x7f38f415b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38f415ac10 0x7f38f415b0c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f38f40b4c00 tx=0x7f38ec00f580 comp rx=0 tx=0).stop 2024-05-27T00:38:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 shutdown_connections 2024-05-27T00:38:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38f415ac10 0x7f38f415b0c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f38f40b5140 0x7f38f40cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 --2- 172.21.15.153:0/3698361210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38e4004800 0x7f38f40b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 >> 172.21.15.153:0/3698361210 conn(0x7f38f40d1f60 msgr2=0x7f38f415ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 shutdown_connections 2024-05-27T00:38:57.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.371+0000 7f38fae38640 1 -- 172.21.15.153:0/3698361210 wait complete. 2024-05-27T00:38:57.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 66cc4ce4-d74b-4952-8f95-f6b7d2c1c562 1 2024-05-27T00:38:57.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.521+0000 7f18776d4640 1 Processor -- start 2024-05-27T00:38:57.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.521+0000 7f18776d4640 1 -- start start 2024-05-27T00:38:57.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f18776d4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f18776d4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f18700b5f80 0x7f18700af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f18776d4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18700b4cc0 con 0x7f187015e190 2024-05-27T00:38:57.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f18776d4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f18700ae700 con 0x7f18700b5f80 2024-05-27T00:38:57.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f18776d4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f18700b4f60 con 0x7f18700ae220 2024-05-27T00:38:57.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:43986/0 (socket says 172.21.15.153:43986) 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 -- 172.21.15.153:0/3988930289 learned_addr learned my addr 172.21.15.153:0/3988930289 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 -- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 msgr2=0x7f18700b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 -- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 msgr2=0x7f18700b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1875c71640 1 --2- 172.21.15.153:0/3988930289 >> v2:172.21.15.111:3300/0 conn(0x7f18700b5f80 0x7f18700af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1874c6f640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1867fff640 1 -- 172.21.15.153:0/3988930289 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829586993 0 0) 0x7f18700b4cc0 con 0x7f187015e190 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1867fff640 1 -- 172.21.15.153:0/3988930289 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f185c003520 con 0x7f187015e190 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1875c71640 1 -- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 msgr2=0x7f18700b5a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1875c71640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1875c71640 1 -- 172.21.15.153:0/3988930289 >> v1:172.21.15.111:6789/0 conn(0x7f187015e190 legacy=0x7f187006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.522+0000 7f1875c71640 1 -- 172.21.15.153:0/3988930289 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18700b5240 con 0x7f18700b5f80 2024-05-27T00:38:57.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.523+0000 7f1875c71640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700b5f80 0x7f18700af2d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1868009dc0 tx=0x7f186802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df40e0033b660101 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.523+0000 7f1867fff640 1 -- 172.21.15.153:0/3988930289 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186802ae60 con 0x7f18700b5f80 2024-05-27T00:38:57.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.523+0000 7f1867fff640 1 -- 172.21.15.153:0/3988930289 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1868007740 con 0x7f18700b5f80 2024-05-27T00:38:57.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.523+0000 7f1867fff640 1 -- 172.21.15.153:0/3988930289 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868007b10 con 0x7f18700b5f80 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 -- 172.21.15.153:0/3988930289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700b5f80 msgr2=0x7f18700af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700b5f80 0x7f18700af2d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1868009dc0 tx=0x7f186802a680 comp rx=0 tx=0).stop 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 -- 172.21.15.153:0/3988930289 shutdown_connections 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700b5f80 0x7f18700af2d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 --2- 172.21.15.153:0/3988930289 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f18700ae220 0x7f18700b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 -- 172.21.15.153:0/3988930289 >> 172.21.15.153:0/3988930289 conn(0x7f187006bc20 msgr2=0x7f18700b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 -- 172.21.15.153:0/3988930289 shutdown_connections 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.524+0000 7f18776d4640 1 -- 172.21.15.153:0/3988930289 wait complete. 2024-05-27T00:38:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.525+0000 7f18776d4640 1 Processor -- start 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.525+0000 7f18776d4640 1 -- start start 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f18776d4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700ae220 0x7f1870154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f18776d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18700b5f80 0x7f1870144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f18776d4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f18776d4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f187015e570 con 0x7f18700ae220 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f18776d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f187006e340 con 0x7f18700b5f80 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f18776d4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f187015e6f0 con 0x7f187015e190 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1874c6f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18700b5f80 0x7f1870144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:43990/0 (socket says 172.21.15.153:43990) 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 -- 172.21.15.153:0/1931215244 learned_addr learned my addr 172.21.15.153:0/1931215244 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875470640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700ae220 0x7f1870154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 -- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 msgr2=0x7f1870141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 -- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 msgr2=0x7f1870141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1875c71640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1874c6f640 1 -- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 msgr2=0x7f1870141900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1874c6f640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1874c6f640 1 -- 172.21.15.153:0/1931215244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700ae220 msgr2=0x7f1870154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1874c6f640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700ae220 0x7f1870154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.526+0000 7f1874c6f640 1 -- 172.21.15.153:0/1931215244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1870145210 con 0x7f18700b5f80 2024-05-27T00:38:57.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f1874c6f640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18700b5f80 0x7f1870144cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f186000e7e0 tx=0x7f186000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860017070 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1860004020 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1870142360 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186000f040 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1870142920 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f186000c6d0 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.527+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f1860013410 con 0x7f18700b5f80 2024-05-27T00:38:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.528+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1870068680 con 0x7f18700b5f80 2024-05-27T00:38:57.533 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:57.528+0000 7f5fb4960040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T00:38:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.530+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1870145210 con 0x7f18700b5f80 2024-05-27T00:38:57.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.600+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "66cc4ce4-d74b-4952-8f95-f6b7d2c1c562", "id": 1} v 0) -- 0x7f1870074c80 con 0x7f18700b5f80 2024-05-27T00:38:57.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.607+0000 7f1865ffb640 1 -- 172.21.15.153:0/1931215244 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66cc4ce4-d74b-4952-8f95-f6b7d2c1c562", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1860003610 con 0x7f18700b5f80 2024-05-27T00:38:57.608 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18700b5f80 msgr2=0x7f1870144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18700b5f80 0x7f1870144cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f186000e7e0 tx=0x7f186000eca0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 shutdown_connections 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f187015e190 0x7f1870141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18700b5f80 0x7f1870144cd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 --2- 172.21.15.153:0/1931215244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18700ae220 0x7f1870154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 >> 172.21.15.153:0/1931215244 conn(0x7f187006bc20 msgr2=0x7f187006c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.608+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 shutdown_connections 2024-05-27T00:38:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.609+0000 7f18776d4640 1 -- 172.21.15.153:0/1931215244 wait complete. 2024-05-27T00:38:57.621 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 83c870e7-b431-4b2e-9cc1-dd7d235821c8 2 2024-05-27T00:38:57.689 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:57.688+0000 7f5fb4960040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T00:38:57.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.722+0000 7fbc5cd34640 1 Processor -- start 2024-05-27T00:38:57.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.722+0000 7fbc5cd34640 1 -- start start 2024-05-27T00:38:57.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc5cd34640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc581459c0 0x7fbc580c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc5cd34640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc580d19f0 0x7fbc580ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc5cd34640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc580c7700 con 0x7fbc580ceeb0 2024-05-27T00:38:57.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc5cd34640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fbc580c79e0 con 0x7fbc581459c0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc5cd34640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbc580c7cc0 con 0x7fbc580d19f0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc55d9b640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc580d19f0 0x7fbc580ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc55d9b640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc580d19f0 0x7fbc580ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55146/0 (socket says 172.21.15.153:55146) 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc55d9b640 1 -- 172.21.15.153:0/2824044536 learned_addr learned my addr 172.21.15.153:0/2824044536 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.723+0000 7fbc5659c640 1 --2- 172.21.15.153:0/2824044536 >> v2:172.21.15.111:3300/0 conn(0x7fbc581459c0 0x7fbc580c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc55d9b640 1 -- 172.21.15.153:0/2824044536 >> v2:172.21.15.111:3300/0 conn(0x7fbc581459c0 msgr2=0x7fbc580c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc55d9b640 1 --2- 172.21.15.153:0/2824044536 >> v2:172.21.15.111:3300/0 conn(0x7fbc581459c0 0x7fbc580c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc55d9b640 1 -- 172.21.15.153:0/2824044536 >> v1:172.21.15.111:6789/0 conn(0x7fbc580ceeb0 legacy=0x7fbc5815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc55d9b640 1 -- 172.21.15.153:0/2824044536 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc580c7fa0 con 0x7fbc580d19f0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc55d9b640 1 --2- 172.21.15.153:0/2824044536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580ce970 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc580b3880 tx=0x7fbc4c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a3503cdf45d56f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc5559a640 1 -- 172.21.15.153:0/2824044536 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c037070 con 0x7fbc580d19f0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc5559a640 1 -- 172.21.15.153:0/2824044536 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c032440 con 0x7fbc580d19f0 2024-05-27T00:38:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.724+0000 7fbc5559a640 1 -- 172.21.15.153:0/2824044536 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c032810 con 0x7fbc580d19f0 2024-05-27T00:38:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 -- 172.21.15.153:0/2824044536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 msgr2=0x7fbc580ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/2824044536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580ce970 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc580b3880 tx=0x7fbc4c02a1e0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 -- 172.21.15.153:0/2824044536 shutdown_connections 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/2824044536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580ce970 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/2824044536 >> v2:172.21.15.111:3300/0 conn(0x7fbc581459c0 0x7fbc580c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 -- 172.21.15.153:0/2824044536 >> 172.21.15.153:0/2824044536 conn(0x7fbc580d1f60 msgr2=0x7fbc580d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 -- 172.21.15.153:0/2824044536 shutdown_connections 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.725+0000 7fbc5cd34640 1 -- 172.21.15.153:0/2824044536 wait complete. 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.726+0000 7fbc5cd34640 1 Processor -- start 2024-05-27T00:38:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 -- start start 2024-05-27T00:38:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 0x7fbc581427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc581459c0 0x7fbc580b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58145f20 con 0x7fbc581459c0 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58145da0 con 0x7fbc580d19f0 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5cd34640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbc580cd960 con 0x7fbc580ceeb0 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc55d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5659c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 0x7fbc581427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc55d9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55158/0 (socket says 172.21.15.153:55158) 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc55d9b640 1 -- 172.21.15.153:0/4211966818 learned_addr learned my addr 172.21.15.153:0/4211966818 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc56d9d640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc581459c0 0x7fbc580b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5659c640 1 -- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 msgr2=0x7fbc581427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5659c640 1 -- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 msgr2=0x7fbc581427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.727+0000 7fbc5659c640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 0x7fbc581427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc5659c640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 0x7fbc581427c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc55d9b640 1 -- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 msgr2=0x7fbc581427c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc55d9b640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 0x7fbc581427c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc55d9b640 1 -- 172.21.15.153:0/4211966818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc581459c0 msgr2=0x7fbc580b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc55d9b640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc581459c0 0x7fbc580b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc55d9b640 1 -- 172.21.15.153:0/4211966818 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc580b5130 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc55d9b640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580b61d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c02a220 tx=0x7fbc4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c037040 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c042070 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc580b5410 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c030880 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc580b59d0 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc4c030a20 con 0x7fbc580d19f0 2024-05-27T00:38:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.728+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fbc4c03b430 con 0x7fbc580d19f0 2024-05-27T00:38:57.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.729+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc58068680 con 0x7fbc580d19f0 2024-05-27T00:38:57.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.731+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbc4c047460 con 0x7fbc580d19f0 2024-05-27T00:38:57.742 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:57.742+0000 7f5fb4960040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T00:38:57.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.803+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "83c870e7-b431-4b2e-9cc1-dd7d235821c8", "id": 2} v 0) -- 0x7fbc580ce3c0 con 0x7fbc580d19f0 2024-05-27T00:38:57.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.808+0000 7fbc46ffd640 1 -- 172.21.15.153:0/4211966818 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83c870e7-b431-4b2e-9cc1-dd7d235821c8", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fbc4c047af0 con 0x7fbc580d19f0 2024-05-27T00:38:57.809 INFO:teuthology.orchestra.run.smithi153.stdout:2 2024-05-27T00:38:57.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 msgr2=0x7fbc580b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580b61d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c02a220 tx=0x7fbc4c0046c0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 shutdown_connections 2024-05-27T00:38:57.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc581459c0 0x7fbc580b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc580d19f0 0x7fbc580b61d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 --2- 172.21.15.153:0/4211966818 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbc580ceeb0 0x7fbc581427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 >> 172.21.15.153:0/4211966818 conn(0x7fbc580d1f60 msgr2=0x7fbc580cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 shutdown_connections 2024-05-27T00:38:57.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.809+0000 7fbc5cd34640 1 -- 172.21.15.153:0/4211966818 wait complete. 2024-05-27T00:38:57.821 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 34090321-f9cd-47ea-bfce-db5326e5d4a9 3 2024-05-27T00:38:57.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.922+0000 7fa7b9390640 1 Processor -- start 2024-05-27T00:38:57.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.922+0000 7fa7b9390640 1 -- start start 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.923+0000 7fa7b9390640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa7b40ae220 0x7fa7b40b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.923+0000 7fa7b9390640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7b40b5f80 0x7fa7b40af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.923+0000 7fa7b9390640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b40b4cc0 con 0x7fa7b415e190 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.923+0000 7fa7b9390640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b40ae700 con 0x7fa7b40ae220 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.923+0000 7fa7b9390640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b40b4f60 con 0x7fa7b40b5f80 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b37fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7b40b5f80 0x7fa7b40af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b2ffd640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa7b415e190 0x7fa7b406dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:36682/0 (socket says 172.21.15.153:36682) 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b2ffd640 1 -- 172.21.15.153:0/192326121 learned_addr learned my addr 172.21.15.153:0/192326121 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-05-27T00:38:57.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7abfff640 1 --2- 172.21.15.153:0/192326121 >> v2:172.21.15.111:3300/0 conn(0x7fa7b40ae220 0x7fa7b40b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b37fe640 1 -- 172.21.15.153:0/192326121 >> v2:172.21.15.111:3300/0 conn(0x7fa7b40ae220 msgr2=0x7fa7b40b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:57.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b37fe640 1 --2- 172.21.15.153:0/192326121 >> v2:172.21.15.111:3300/0 conn(0x7fa7b40ae220 0x7fa7b40b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b37fe640 1 -- 172.21.15.153:0/192326121 >> v1:172.21.15.153:6789/0 conn(0x7fa7b415e190 legacy=0x7fa7b406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b27fc640 1 -- 172.21.15.153:0/192326121 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2572209199 0 0) 0x7fa7b40b4cc0 con 0x7fa7b415e190 2024-05-27T00:38:57.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.924+0000 7fa7b37fe640 1 -- 172.21.15.153:0/192326121 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40b5240 con 0x7fa7b40b5f80 2024-05-27T00:38:57.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b37fe640 1 --2- 172.21.15.153:0/192326121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40af2d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40b3f90 tx=0x7fa7ac02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31538d9e039f7f91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b27fc640 1 -- 172.21.15.153:0/192326121 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac007740 con 0x7fa7b40b5f80 2024-05-27T00:38:57.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b27fc640 1 -- 172.21.15.153:0/192326121 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ac0078e0 con 0x7fa7b40b5f80 2024-05-27T00:38:57.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b27fc640 1 -- 172.21.15.153:0/192326121 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac007cb0 con 0x7fa7b40b5f80 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 -- 172.21.15.153:0/192326121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40b5f80 msgr2=0x7fa7b40af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 --2- 172.21.15.153:0/192326121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40af2d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40b3f90 tx=0x7fa7ac02a5f0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 -- 172.21.15.153:0/192326121 shutdown_connections 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 --2- 172.21.15.153:0/192326121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40af2d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 --2- 172.21.15.153:0/192326121 >> v2:172.21.15.111:3300/0 conn(0x7fa7b40ae220 0x7fa7b40b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 -- 172.21.15.153:0/192326121 >> 172.21.15.153:0/192326121 conn(0x7fa7b406bc20 msgr2=0x7fa7b40b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.925+0000 7fa7b9390640 1 -- 172.21.15.153:0/192326121 shutdown_connections 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.926+0000 7fa7b9390640 1 -- 172.21.15.153:0/192326121 wait complete. 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 Processor -- start 2024-05-27T00:38:57.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 -- start start 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 0x7fa7b414d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 0x7fa7b414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b415e570 con 0x7fa7b40b5f80 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b406e340 con 0x7fa7b40ae220 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b9390640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b415e6f0 con 0x7fa7b415e190 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.927+0000 7fa7b2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 0x7fa7b414d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 0x7fa7b414d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55178/0 (socket says 172.21.15.153:55178) 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 -- 172.21.15.153:0/2824462281 learned_addr learned my addr 172.21.15.153:0/2824462281 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b37fe640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 0x7fa7b414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7abfff640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b37fe640 1 -- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 msgr2=0x7fa7b414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b37fe640 1 -- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 msgr2=0x7fa7b414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b37fe640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 0x7fa7b414e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b37fe640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 0x7fa7b414e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 -- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 msgr2=0x7fa7b414e340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 0x7fa7b414e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 -- 172.21.15.153:0/2824462281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7b40b5f80 msgr2=0x7fa7b40c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 -- 172.21.15.153:0/2824462281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40c58f0 con 0x7fa7b40ae220 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b2ffd640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 0x7fa7b414d1c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c460 tx=0x7fa7a400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4018070 con 0x7fa7b40ae220 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4007500 con 0x7fa7b40ae220 2024-05-27T00:38:57.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b40c5bd0 con 0x7fa7b40ae220 2024-05-27T00:38:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004a20 con 0x7fa7b40ae220 2024-05-27T00:38:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b40c6190 con 0x7fa7b40ae220 2024-05-27T00:38:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7a401b070 con 0x7fa7b40ae220 2024-05-27T00:38:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.928+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa7a4012890 con 0x7fa7b40ae220 2024-05-27T00:38:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.929+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b40c6390 con 0x7fa7b40ae220 2024-05-27T00:38:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:57.931+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa7a4012400 con 0x7fa7b40ae220 2024-05-27T00:38:57.986 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:57.985+0000 7f5fb4960040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T00:38:58.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.003+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "34090321-f9cd-47ea-bfce-db5326e5d4a9", "id": 3} v 0) -- 0x7fa7b4068680 con 0x7fa7b40ae220 2024-05-27T00:38:58.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.009+0000 7fa7ab7fe640 1 -- 172.21.15.153:0/2824462281 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "34090321-f9cd-47ea-bfce-db5326e5d4a9", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa7a4004bc0 con 0x7fa7b40ae220 2024-05-27T00:38:58.010 INFO:teuthology.orchestra.run.smithi153.stdout:3 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 msgr2=0x7fa7b414d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 0x7fa7b414d1c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c460 tx=0x7fa7a400c920 comp rx=0 tx=0).stop 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 shutdown_connections 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa7b415e190 0x7fa7b414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7b40b5f80 0x7fa7b40c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 --2- 172.21.15.153:0/2824462281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7b40ae220 0x7fa7b414d1c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 >> 172.21.15.153:0/2824462281 conn(0x7fa7b406bc20 msgr2=0x7fa7b40aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 shutdown_connections 2024-05-27T00:38:58.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.010+0000 7fa7b9390640 1 -- 172.21.15.153:0/2824462281 wait complete. 2024-05-27T00:38:58.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d 4 2024-05-27T00:38:58.031 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:58.031+0000 7f5fb4960040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T00:38:58.058 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.057+0000 7fa9b3932040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T00:38:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.121+0000 7fa9a2b09640 1 Processor -- start 2024-05-27T00:38:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.121+0000 7fa9a2b09640 1 -- start start 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a2b09640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa99c15bed0 0x7fa99c0b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a2b09640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa99c15f1b0 0x7fa99c0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a2b09640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c15f680 con 0x7fa99c06dfc0 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a2b09640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c15f960 con 0x7fa99c15bed0 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a2b09640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c06d4e0 con 0x7fa99c15f1b0 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a10a6640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa99c15f1b0 0x7fa99c0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa99bfff640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa99c06dfc0 0x7fa99c15eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:36698/0 (socket says 172.21.15.153:36698) 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a10a6640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa99c15f1b0 0x7fa99c0b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55182/0 (socket says 172.21.15.153:55182) 2024-05-27T00:38:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa99bfff640 1 -- 172.21.15.153:0/3093034442 learned_addr learned my addr 172.21.15.153:0/3093034442 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a08a5640 1 --2- 172.21.15.153:0/3093034442 >> v2:172.21.15.111:3300/0 conn(0x7fa99c15bed0 0x7fa99c0b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a10a6640 1 -- 172.21.15.153:0/3093034442 >> v2:172.21.15.111:3300/0 conn(0x7fa99c15bed0 msgr2=0x7fa99c0b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a10a6640 1 --2- 172.21.15.153:0/3093034442 >> v2:172.21.15.111:3300/0 conn(0x7fa99c15bed0 0x7fa99c0b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a10a6640 1 -- 172.21.15.153:0/3093034442 >> v1:172.21.15.153:6789/0 conn(0x7fa99c06dfc0 legacy=0x7fa99c15eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa9a10a6640 1 -- 172.21.15.153:0/3093034442 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c06d7c0 con 0x7fa99c15f1b0 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.122+0000 7fa99b7fe640 1 -- 172.21.15.153:0/3093034442 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939134360 0 0) 0x7fa99c15f680 con 0x7fa99c06dfc0 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.123+0000 7fa9a10a6640 1 --2- 172.21.15.153:0/3093034442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15f1b0 0x7fa99c0b5c70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa99c158ff0 tx=0x7fa99402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba1f15843501d1e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.123+0000 7fa99b7fe640 1 -- 172.21.15.153:0/3093034442 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994007740 con 0x7fa99c15f1b0 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.123+0000 7fa99b7fe640 1 -- 172.21.15.153:0/3093034442 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9940078e0 con 0x7fa99c15f1b0 2024-05-27T00:38:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.123+0000 7fa99b7fe640 1 -- 172.21.15.153:0/3093034442 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994007cb0 con 0x7fa99c15f1b0 2024-05-27T00:38:58.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3093034442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15f1b0 msgr2=0x7fa99c0b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3093034442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15f1b0 0x7fa99c0b5c70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa99c158ff0 tx=0x7fa99402a5f0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3093034442 shutdown_connections 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3093034442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15f1b0 0x7fa99c0b5c70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3093034442 >> v2:172.21.15.111:3300/0 conn(0x7fa99c15bed0 0x7fa99c0b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3093034442 >> 172.21.15.153:0/3093034442 conn(0x7fa99c073130 msgr2=0x7fa99c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3093034442 shutdown_connections 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.124+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3093034442 wait complete. 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.125+0000 7fa9a2b09640 1 Processor -- start 2024-05-27T00:38:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.125+0000 7fa9a2b09640 1 -- start start 2024-05-27T00:38:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a2b09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa99c06dfc0 0x7fa99c154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a2b09640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 0x7fa99c1406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a2b09640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a2b09640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c15c430 con 0x7fa99c06dfc0 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a2b09640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c15c2b0 con 0x7fa99c15bed0 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a2b09640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c06f810 con 0x7fa99c15f1b0 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 0x7fa99c1406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a10a6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 0x7fa99c1406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55186/0 (socket says 172.21.15.153:55186) 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa99bfff640 1 -- 172.21.15.153:0/3953364605 learned_addr learned my addr 172.21.15.153:0/3953364605 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.126+0000 7fa9a10a6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:44008/0 (socket says 172.21.15.153:44008) 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a08a5640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa99c06dfc0 0x7fa99c154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a10a6640 1 -- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 msgr2=0x7fa99c0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a10a6640 1 -- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 msgr2=0x7fa99c0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a10a6640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a10a6640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa99bfff640 1 -- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 msgr2=0x7fa99c0b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa99bfff640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa99bfff640 1 -- 172.21.15.153:0/3953364605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa99c06dfc0 msgr2=0x7fa99c154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa99bfff640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa99c06dfc0 0x7fa99c154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa99bfff640 1 -- 172.21.15.153:0/3953364605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c154950 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a08a5640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa99c06dfc0 0x7fa99c154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa99bfff640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 0x7fa99c1406b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c4d0 tx=0x7fa98800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988018070 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa988007500 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c154c30 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988004a20 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c1551f0 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa98801b070 con 0x7fa99c15bed0 2024-05-27T00:38:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.127+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fa988012910 con 0x7fa99c15bed0 2024-05-27T00:38:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.129+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c068680 con 0x7fa99c15bed0 2024-05-27T00:38:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.130+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa988012c50 con 0x7fa99c15bed0 2024-05-27T00:38:58.165 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.164+0000 7fa9b3932040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T00:38:58.235 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.234+0000 7fa9b3932040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T00:38:58.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.256+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d", "id": 4} v 0) -- 0x7fa99c0cf050 con 0x7fa99c15bed0 2024-05-27T00:38:58.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.261+0000 7fa9997fa640 1 -- 172.21.15.153:0/3953364605 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fa988004bc0 con 0x7fa99c15bed0 2024-05-27T00:38:58.262 INFO:teuthology.orchestra.run.smithi153.stdout:4 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 msgr2=0x7fa99c1406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 0x7fa99c1406b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c4d0 tx=0x7fa98800c990 comp rx=0 tx=0).stop 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 shutdown_connections 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa99c15f1b0 0x7fa99c0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa99c15bed0 0x7fa99c1406b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 --2- 172.21.15.153:0/3953364605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa99c06dfc0 0x7fa99c154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.262+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 >> 172.21.15.153:0/3953364605 conn(0x7fa99c073130 msgr2=0x7fa99c15eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.263+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 shutdown_connections 2024-05-27T00:38:58.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.263+0000 7fa9a2b09640 1 -- 172.21.15.153:0/3953364605 wait complete. 2024-05-27T00:38:58.274 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 1be914ee-b413-449c-8453-a55c28af7d70 5 2024-05-27T00:38:58.410 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.409+0000 7fa9b3932040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T00:38:58.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.414+0000 7f8993dc1640 1 Processor -- start 2024-05-27T00:38:58.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.415+0000 7f8993dc1640 1 -- start start 2024-05-27T00:38:58.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8993dc1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 0x7f898c0b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8993dc1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f898c06dfc0 0x7f898c0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8993dc1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898c15f680 con 0x7f898c15eaa0 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8993dc1640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f898c15f960 con 0x7f898c06dfc0 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8993dc1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f898c06d4e0 con 0x7f898c15bed0 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f899135c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f898c06dfc0 0x7f898c0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f899135c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f898c06dfc0 0x7f898c0b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55190/0 (socket says 172.21.15.153:55190) 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f899135c640 1 -- 172.21.15.153:0/2067355663 learned_addr learned my addr 172.21.15.153:0/2067355663 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8991b5d640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 0x7f898c0b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8991b5d640 1 -- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 msgr2=0x7f898c0b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8991b5d640 1 -- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 msgr2=0x7f898c0b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.416+0000 7f8991b5d640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 0x7f898c0b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f8991b5d640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 0x7f898c0b5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f899135c640 1 -- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 msgr2=0x7f898c0b5730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f899135c640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 0x7f898c0b5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f899135c640 1 -- 172.21.15.153:0/2067355663 >> v1:172.21.15.111:6789/0 conn(0x7f898c15eaa0 legacy=0x7f898c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f899135c640 1 -- 172.21.15.153:0/2067355663 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c06d7c0 con 0x7f898c06dfc0 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f899135c640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c0b5c70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a2bb425ede830f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.417+0000 7f8990b5b640 1 -- 172.21.15.153:0/2067355663 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c037070 con 0x7f898c06dfc0 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8990b5b640 1 -- 172.21.15.153:0/2067355663 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c032440 con 0x7f898c06dfc0 2024-05-27T00:38:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8990b5b640 1 -- 172.21.15.153:0/2067355663 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c032810 con 0x7f898c06dfc0 2024-05-27T00:38:58.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8993dc1640 1 -- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 msgr2=0x7f898c0b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8993dc1640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c0b5c70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c02a1c0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8993dc1640 1 -- 172.21.15.153:0/2067355663 shutdown_connections 2024-05-27T00:38:58.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8993dc1640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c0b5c70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8993dc1640 1 --2- 172.21.15.153:0/2067355663 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15bed0 0x7f898c0b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.418+0000 7f8993dc1640 1 -- 172.21.15.153:0/2067355663 >> 172.21.15.153:0/2067355663 conn(0x7f898c073130 msgr2=0x7f898c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.419+0000 7f8993dc1640 1 -- 172.21.15.153:0/2067355663 shutdown_connections 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.419+0000 7f8993dc1640 1 -- 172.21.15.153:0/2067355663 wait complete. 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 Processor -- start 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 -- start start 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f898c15bed0 0x7f898c1372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 0x7f898c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f898c15c430 con 0x7f898c15bed0 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f898c15c2b0 con 0x7f898c06dfc0 2024-05-27T00:38:58.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8993dc1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f898c15ef30 con 0x7f898c15eaa0 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8991b5d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f899235e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 0x7f898c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8991b5d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55206/0 (socket says 172.21.15.153:55206) 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.420+0000 7f8991b5d640 1 -- 172.21.15.153:0/594398372 learned_addr learned my addr 172.21.15.153:0/594398372 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f899135c640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f898c15bed0 0x7f898c1372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f899235e640 1 -- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 msgr2=0x7f898c155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f899235e640 1 -- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 msgr2=0x7f898c155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f899235e640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 0x7f898c155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f899235e640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 0x7f898c155b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8991b5d640 1 -- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 msgr2=0x7f898c155b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8991b5d640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 0x7f898c155b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8991b5d640 1 -- 172.21.15.153:0/594398372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f898c15bed0 msgr2=0x7f898c1372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8991b5d640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f898c15bed0 0x7f898c1372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8991b5d640 1 -- 172.21.15.153:0/594398372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c138240 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8991b5d640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c14c110 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f898800d7d0 tx=0x7f898800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898800b6c0 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8988019070 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c137c80 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898800bac0 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1571a0 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f898800bc60 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.421+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f8988014aa0 con 0x7f898c06dfc0 2024-05-27T00:38:58.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.422+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c068680 con 0x7f898c06dfc0 2024-05-27T00:38:58.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.424+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8988014d80 con 0x7f898c06dfc0 2024-05-27T00:38:58.465 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.464+0000 7fa9b3932040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T00:38:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.496+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1be914ee-b413-449c-8453-a55c28af7d70", "id": 5} v 0) -- 0x7f898c166c20 con 0x7f898c06dfc0 2024-05-27T00:38:58.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.502+0000 7f89827fc640 1 -- 172.21.15.153:0/594398372 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1be914ee-b413-449c-8453-a55c28af7d70", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f89880145f0 con 0x7f898c06dfc0 2024-05-27T00:38:58.503 INFO:teuthology.orchestra.run.smithi153.stdout:5 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 msgr2=0x7f898c14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c14c110 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f898800d7d0 tx=0x7f898800dc90 comp rx=0 tx=0).stop 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 shutdown_connections 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f898c15eaa0 0x7f898c155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f898c15bed0 0x7f898c1372a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 --2- 172.21.15.153:0/594398372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f898c06dfc0 0x7f898c14c110 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 >> 172.21.15.153:0/594398372 conn(0x7f898c073130 msgr2=0x7f898c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 shutdown_connections 2024-05-27T00:38:58.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.503+0000 7f8993dc1640 1 -- 172.21.15.153:0/594398372 wait complete. 2024-05-27T00:38:58.514 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new c88ab04d-9802-4926-b3e4-38a8d055e63d 6 2024-05-27T00:38:58.520 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.519+0000 7fa9b3932040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T00:38:58.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.613+0000 7f2234e9a640 1 Processor -- start 2024-05-27T00:38:58.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.613+0000 7f2234e9a640 1 -- start start 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f2234e9a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22300b3630 0x7f22300b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f2234e9a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 0x7f2230161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22300b3630 0x7f22300b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f2234e9a640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f223006cc70 con 0x7f2230162390 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 0x7f2230161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22300b3630 0x7f22300b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.153:34768/0 (socket says 172.21.15.153:34768) 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222e59c640 1 -- 172.21.15.153:0/3126574249 learned_addr learned my addr 172.21.15.153:0/3126574249 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:58.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f22300d1e50 con 0x7f22300b3630 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f22300d2130 con 0x7f22300b77e0 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222dd9b640 1 -- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 msgr2=0x7f2230161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222dd9b640 1 -- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 msgr2=0x7f2230161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.614+0000 7f222dd9b640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 0x7f2230161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.615+0000 7f222dd9b640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 0x7f2230161e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.615+0000 7f222e59c640 1 -- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 msgr2=0x7f2230161e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.615+0000 7f222e59c640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 0x7f2230161e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.615+0000 7f222e59c640 1 -- 172.21.15.153:0/3126574249 >> v1:172.21.15.153:6789/0 conn(0x7f2230162390 legacy=0x7f22300d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.615+0000 7f222e59c640 1 -- 172.21.15.153:0/3126574249 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22300d2410 con 0x7f22300b3630 2024-05-27T00:38:58.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.615+0000 7f222e59c640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f22300b72a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2220009870 tx=0x7f222002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c4649b8242f81a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.616+0000 7f222d59a640 1 -- 172.21.15.153:0/3126574249 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220037070 con 0x7f22300b3630 2024-05-27T00:38:58.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.616+0000 7f222d59a640 1 -- 172.21.15.153:0/3126574249 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2220032440 con 0x7f22300b3630 2024-05-27T00:38:58.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.616+0000 7f222d59a640 1 -- 172.21.15.153:0/3126574249 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220032810 con 0x7f22300b3630 2024-05-27T00:38:58.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 msgr2=0x7f22300b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f22300b72a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2220009870 tx=0x7f222002a270 comp rx=0 tx=0).stop 2024-05-27T00:38:58.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 shutdown_connections 2024-05-27T00:38:58.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f22300b77e0 0x7f2230161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 --2- 172.21.15.153:0/3126574249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f22300b72a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 >> 172.21.15.153:0/3126574249 conn(0x7f22300d42c0 msgr2=0x7f22300d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 shutdown_connections 2024-05-27T00:38:58.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.617+0000 7f2234e9a640 1 -- 172.21.15.153:0/3126574249 wait complete. 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.618+0000 7f2234e9a640 1 Processor -- start 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.618+0000 7f2234e9a640 1 -- start start 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f2234e9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f223015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f2234e9a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 0x7f2230152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f2234e9a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f2234e9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22300d34a0 con 0x7f22300b3630 2024-05-27T00:38:58.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f2234e9a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f22300d3320 con 0x7f22300b77e0 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f2234e9a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f22300d3620 con 0x7f2230162390 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 0x7f2230152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222ed9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 0x7f2230152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55212/0 (socket says 172.21.15.153:55212) 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222ed9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:44054/0 (socket says 172.21.15.153:44054) 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 -- 172.21.15.153:0/1058928029 learned_addr learned my addr 172.21.15.153:0/1058928029 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222e59c640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f223015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222ed9d640 1 -- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 msgr2=0x7f22300cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222ed9d640 1 -- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 msgr2=0x7f22300cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222ed9d640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222ed9d640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 -- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 msgr2=0x7f22300cdbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:58.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 -- 172.21.15.153:0/1058928029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 msgr2=0x7f223015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f223015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 -- 172.21.15.153:0/1058928029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22300c9bc0 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.619+0000 7f222dd9b640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 0x7f2230152430 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f222400d7d0 tx=0x7f222400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222400b6c0 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2224019070 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22300c9ea0 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222400bac0 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22300ca740 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f222400bc60 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.620+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f2224014b10 con 0x7f22300b77e0 2024-05-27T00:38:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.621+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22300b9f70 con 0x7f22300b77e0 2024-05-27T00:38:58.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.622+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f22300b9f70 con 0x7f22300b77e0 2024-05-27T00:38:58.641 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:58.640+0000 7fa9b3932040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T00:38:58.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.695+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c88ab04d-9802-4926-b3e4-38a8d055e63d", "id": 6} v 0) -- 0x7f2230068680 con 0x7f22300b77e0 2024-05-27T00:38:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.703+0000 7f221effd640 1 -- 172.21.15.153:0/1058928029 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c88ab04d-9802-4926-b3e4-38a8d055e63d", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f2224014520 con 0x7f22300b77e0 2024-05-27T00:38:58.704 INFO:teuthology.orchestra.run.smithi153.stdout:6 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 msgr2=0x7f2230152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 0x7f2230152430 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f222400d7d0 tx=0x7f222400dc90 comp rx=0 tx=0).stop 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 shutdown_connections 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2230162390 0x7f22300cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22300b77e0 0x7f2230152430 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 --2- 172.21.15.153:0/1058928029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22300b3630 0x7f223015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 >> 172.21.15.153:0/1058928029 conn(0x7f22300d42c0 msgr2=0x7f22300b7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 shutdown_connections 2024-05-27T00:38:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.704+0000 7f2234e9a640 1 -- 172.21.15.153:0/1058928029 wait complete. 2024-05-27T00:38:58.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd new 1356774a-a6a9-4ce5-be06-a153448df9db 7 2024-05-27T00:38:58.748 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:58.747+0000 7f5fb4960040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T00:38:58.833 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:58.832+0000 7f5fb4960040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T00:38:58.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.856+0000 7f9193612640 1 Processor -- start 2024-05-27T00:38:58.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.856+0000 7f9193612640 1 -- start start 2024-05-27T00:38:58.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9193612640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f918c0d05a0 0x7f918c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9193612640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f918c144c10 con 0x7f918c0b54f0 2024-05-27T00:38:58.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9193612640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f918c06f510 con 0x7f918c144830 2024-05-27T00:38:58.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9193612640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f918c0b5940 con 0x7f918c0d05a0 2024-05-27T00:38:58.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f91913ae640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f918c144830 0x7f918c0705b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:36718/0 (socket says 172.21.15.153:36718) 2024-05-27T00:38:58.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f91913ae640 1 -- 172.21.15.153:0/2369319837 learned_addr learned my addr 172.21.15.153:0/2369319837 (peer_addr_for_me v1:172.21.15.153:0/0) 2024-05-27T00:38:58.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9190bad640 1 --2- 172.21.15.153:0/2369319837 >> v2:172.21.15.111:3300/0 conn(0x7f918c0d05a0 0x7f918c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4055501707 0 0) 0x7f918c06f510 con 0x7f918c144830 2024-05-27T00:38:58.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9178003520 con 0x7f918c144830 2024-05-27T00:38:58.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523897646 0 0) 0x7f918c144c10 con 0x7f918c0b54f0 2024-05-27T00:38:58.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f918c06f510 con 0x7f918c0b54f0 2024-05-27T00:38:58.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9190bad640 1 -- 172.21.15.153:0/2369319837 >> v1:172.21.15.153:6789/0 conn(0x7f918c144830 legacy=0x7f918c0705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9190bad640 1 -- 172.21.15.153:0/2369319837 >> v1:172.21.15.111:6789/0 conn(0x7f918c0b54f0 legacy=0x7f918c15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.857+0000 7f9190bad640 1 -- 172.21.15.153:0/2369319837 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c06f7f0 con 0x7f918c0d05a0 2024-05-27T00:38:58.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.858+0000 7f9190bad640 1 --2- 172.21.15.153:0/2369319837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0d05a0 0x7f918c0b4fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f917c0097a0 tx=0x7f917c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ba24bd369d0118 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.858+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c037070 con 0x7f918c0d05a0 2024-05-27T00:38:58.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.858+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c032440 con 0x7f918c0d05a0 2024-05-27T00:38:58.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.858+0000 7f9183fff640 1 -- 172.21.15.153:0/2369319837 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c032810 con 0x7f918c0d05a0 2024-05-27T00:38:58.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 -- 172.21.15.153:0/2369319837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0d05a0 msgr2=0x7f918c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 --2- 172.21.15.153:0/2369319837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0d05a0 0x7f918c0b4fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f917c0097a0 tx=0x7f917c02a230 comp rx=0 tx=0).stop 2024-05-27T00:38:58.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 -- 172.21.15.153:0/2369319837 shutdown_connections 2024-05-27T00:38:58.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 --2- 172.21.15.153:0/2369319837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0d05a0 0x7f918c0b4fb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 -- 172.21.15.153:0/2369319837 >> 172.21.15.153:0/2369319837 conn(0x7f918c072250 msgr2=0x7f918c072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 -- 172.21.15.153:0/2369319837 shutdown_connections 2024-05-27T00:38:58.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.859+0000 7f9193612640 1 -- 172.21.15.153:0/2369319837 wait complete. 2024-05-27T00:38:58.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.860+0000 7f9193612640 1 Processor -- start 2024-05-27T00:38:58.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.860+0000 7f9193612640 1 -- start start 2024-05-27T00:38:58.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9193612640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0b54f0 0x7f918c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9193612640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9193612640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 0x7f918c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:58.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9193612640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f918c06fdb0 con 0x7f918c0b54f0 2024-05-27T00:38:58.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9193612640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f918c070120 con 0x7f918c144830 2024-05-27T00:38:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9193612640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f918c0d0a00 con 0x7f918c0d05a0 2024-05-27T00:38:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 0x7f918c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.153:44070/0 (socket says 172.21.15.153:44070) 2024-05-27T00:38:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 -- 172.21.15.153:0/3189848118 learned_addr learned my addr 172.21.15.153:0/3189848118 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-05-27T00:38:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 0x7f918c0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55220/0 (socket says 172.21.15.153:55220) 2024-05-27T00:38:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f91913ae640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0b54f0 0x7f918c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 -- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 msgr2=0x7f918c0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-27T00:38:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 -- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 msgr2=0x7f918c0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:38:58.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:38:58.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9190bad640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:38:58.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 -- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 msgr2=0x7f918c0b2090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:38:58.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 -- 172.21.15.153:0/3189848118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0b54f0 msgr2=0x7f918c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0b54f0 0x7f918c15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f9191baf640 1 -- 172.21.15.153:0/3189848118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c0b0920 con 0x7f918c144830 2024-05-27T00:38:58.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.861+0000 7f91913ae640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0b54f0 0x7f918c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:38:58.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9191baf640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 0x7f918c0b25d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f918400c6c0 tx=0x7f918400cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184003b70 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9184003d10 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c0b0c00 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184005310 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c0b11c0 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91840054b0 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.862+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f9184012bc0 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.863+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c068680 con 0x7f918c144830 2024-05-27T00:38:58.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.865+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9184012560 con 0x7f918c144830 2024-05-27T00:38:58.879 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:58.874+0000 7f5fb4960040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T00:38:58.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.938+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1356774a-a6a9-4ce5-be06-a153448df9db", "id": 7} v 0) -- 0x7f918c0aab30 con 0x7f918c144830 2024-05-27T00:38:58.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.946+0000 7f9181ffb640 1 -- 172.21.15.153:0/3189848118 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1356774a-a6a9-4ce5-be06-a153448df9db", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9184012740 con 0x7f918c144830 2024-05-27T00:38:58.947 INFO:teuthology.orchestra.run.smithi153.stdout:7 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 msgr2=0x7f918c0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 0x7f918c0b25d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f918400c6c0 tx=0x7f918400cb80 comp rx=0 tx=0).stop 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 shutdown_connections 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f918c144830 0x7f918c0b25d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f918c0d05a0 0x7f918c0b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 --2- 172.21.15.153:0/3189848118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f918c0b54f0 0x7f918c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 >> 172.21.15.153:0/3189848118 conn(0x7f918c072250 msgr2=0x7f918c0a9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:58.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 shutdown_connections 2024-05-27T00:38:58.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:38:58.947+0000 7f9193612640 1 -- 172.21.15.153:0/3189848118 wait complete. 2024-05-27T00:38:58.959 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-27T00:38:58.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-27T00:38:58.962 INFO:tasks.ceph.osd.0:Started 2024-05-27T00:38:58.962 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-27T00:38:58.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-27T00:38:58.965 INFO:tasks.ceph.osd.1:Started 2024-05-27T00:38:58.965 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-27T00:38:58.965 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-27T00:38:58.967 INFO:tasks.ceph.osd.2:Started 2024-05-27T00:38:58.968 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-27T00:38:58.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-27T00:38:58.970 INFO:tasks.ceph.osd.3:Started 2024-05-27T00:38:58.970 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-27T00:38:58.971 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-27T00:38:58.972 INFO:tasks.ceph.osd.4:Started 2024-05-27T00:38:58.973 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-27T00:38:58.973 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-27T00:38:58.977 INFO:tasks.ceph.osd.5:Started 2024-05-27T00:38:58.977 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-27T00:38:58.977 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-27T00:38:58.979 INFO:tasks.ceph.osd.6:Started 2024-05-27T00:38:58.979 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-27T00:38:58.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-27T00:38:58.981 INFO:tasks.ceph.osd.7:Started 2024-05-27T00:38:58.981 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:38:59.056 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:59.055+0000 7fa9b3932040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T00:38:59.069 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.069+0000 7f5fb4960040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T00:38:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.108+0000 7f7d524da640 1 Processor -- start 2024-05-27T00:38:59.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.108+0000 7f7d524da640 1 -- start start 2024-05-27T00:38:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d524da640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d4c15a240 0x7f7d4c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:59.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d524da640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 0x7f7d4c071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d524da640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d4c0b5160 con 0x7f7d4c0cf260 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d524da640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4c0b4670 con 0x7f7d4c0cf8b0 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d524da640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4c0b48c0 con 0x7f7d4c15a240 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d43fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 0x7f7d4c071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d50a77640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7d4c0cf260 0x7f7d4c15df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:58670/0 (socket says 172.21.15.111:58670) 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d50a77640 1 -- 172.21.15.111:0/3921713083 learned_addr learned my addr 172.21.15.111:0/3921713083 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d4bfff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d4c15a240 0x7f7d4c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d43fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 0x7f7d4c071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49240/0 (socket says 172.21.15.111:49240) 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d4bfff640 1 -- 172.21.15.111:0/3921713083 >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 msgr2=0x7f7d4c071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d4bfff640 1 --2- 172.21.15.111:0/3921713083 >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 0x7f7d4c071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d4bfff640 1 -- 172.21.15.111:0/3921713083 >> v1:172.21.15.153:6789/0 conn(0x7f7d4c0cf260 legacy=0x7f7d4c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d4bfff640 1 -- 172.21.15.111:0/3921713083 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c0b4ba0 con 0x7f7d4c15a240 2024-05-27T00:38:59.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d43fff640 1 --2- 172.21.15.111:0/3921713083 >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 0x7f7d4c071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:38:59.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.109+0000 7f7d4b7fe640 1 -- 172.21.15.111:0/3921713083 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636195024 0 0) 0x7f7d4c0b5160 con 0x7f7d4c0cf260 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.110+0000 7f7d4bfff640 1 --2- 172.21.15.111:0/3921713083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c15a240 0x7f7d4c0719e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0b47f0 tx=0x7f7d3c02c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f23c08dcbde7717e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.110+0000 7f7d4b7fe640 1 -- 172.21.15.111:0/3921713083 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c038070 con 0x7f7d4c15a240 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.110+0000 7f7d4b7fe640 1 -- 172.21.15.111:0/3921713083 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3c008df0 con 0x7f7d4c15a240 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.110+0000 7f7d4b7fe640 1 -- 172.21.15.111:0/3921713083 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c00a2f0 con 0x7f7d4c15a240 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 -- 172.21.15.111:0/3921713083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c15a240 msgr2=0x7f7d4c0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 --2- 172.21.15.111:0/3921713083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c15a240 0x7f7d4c0719e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0b47f0 tx=0x7f7d3c02c490 comp rx=0 tx=0).stop 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 -- 172.21.15.111:0/3921713083 shutdown_connections 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 --2- 172.21.15.111:0/3921713083 >> v2:172.21.15.111:3300/0 conn(0x7f7d4c0cf8b0 0x7f7d4c071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 --2- 172.21.15.111:0/3921713083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c15a240 0x7f7d4c0719e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 -- 172.21.15.111:0/3921713083 >> 172.21.15.111:0/3921713083 conn(0x7f7d4c0b1c00 msgr2=0x7f7d4c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 -- 172.21.15.111:0/3921713083 shutdown_connections 2024-05-27T00:38:59.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.111+0000 7f7d524da640 1 -- 172.21.15.111:0/3921713083 wait complete. 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.112+0000 7f7d524da640 1 Processor -- start 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.112+0000 7f7d524da640 1 -- start start 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d524da640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c0cf260 0x7f7d4c0aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d524da640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d4c0cf8b0 0x7f7d4c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d524da640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 0x7f7d4c1efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d524da640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c15a620 con 0x7f7d4c15a240 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d524da640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4c072580 con 0x7f7d4c0cf260 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d524da640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4c15a7a0 con 0x7f7d4c0cf8b0 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d50a77640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 0x7f7d4c1efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d50a77640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 0x7f7d4c1efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49244/0 (socket says 172.21.15.111:49244) 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d50a77640 1 -- 172.21.15.111:0/3418470792 learned_addr learned my addr 172.21.15.111:0/3418470792 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:38:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d4bfff640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c0cf260 0x7f7d4c0aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d4bfff640 1 -- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d4c0cf8b0 msgr2=0x7f7d4c156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d4bfff640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d4c0cf8b0 0x7f7d4c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d4bfff640 1 -- 172.21.15.111:0/3418470792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 msgr2=0x7f7d4c1efcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d4bfff640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 0x7f7d4c1efcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.113+0000 7f7d4bfff640 1 -- 172.21.15.111:0/3418470792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4c1f06f0 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d50a77640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 0x7f7d4c1efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d4bfff640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c0cf260 0x7f7d4c0aa450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c032000 tx=0x7f7d3c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c043070 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3c033830 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4c1f09d0 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c038040 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4c155e20 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d3c046070 con 0x7f7d4c0cf260 2024-05-27T00:38:59.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.114+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7d3c033c00 con 0x7f7d4c0cf260 2024-05-27T00:38:59.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.115+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4c068680 con 0x7f7d4c0cf260 2024-05-27T00:38:59.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.117+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7d3c03c430 con 0x7f7d4c0cf260 2024-05-27T00:38:59.120 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.120+0000 7f5fb4960040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T00:38:59.174 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.173+0000 7f5fb4960040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T00:38:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.176+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d4c159a60 con 0x7f7d4c0cf260 2024-05-27T00:38:59.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.177+0000 7f7d497fa640 1 -- 172.21.15.111:0/3418470792 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7d3c03c610 con 0x7f7d4c0cf260 2024-05-27T00:38:59.178 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:38:59.178 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:38:58.939697+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:38:59.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c0cf260 msgr2=0x7f7d4c0aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:38:59.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c0cf260 0x7f7d4c0aa450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c032000 tx=0x7f7d3c0046c0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 shutdown_connections 2024-05-27T00:38:59.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d4c15a240 0x7f7d4c1efcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7d4c0cf8b0 0x7f7d4c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 --2- 172.21.15.111:0/3418470792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d4c0cf260 0x7f7d4c0aa450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:38:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 >> 172.21.15.111:0/3418470792 conn(0x7f7d4c0b1c00 msgr2=0x7f7d4c0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:38:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 shutdown_connections 2024-05-27T00:38:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:38:59.178+0000 7f7d524da640 1 -- 172.21.15.111:0/3418470792 wait complete. 2024-05-27T00:38:59.190 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-27T00:38:59.190 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-27T00:38:59.287 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.286+0000 7f5fb4960040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T00:38:59.326 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:59.325+0000 7fa9b3932040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T00:38:59.379 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:59.379+0000 7fa9b3932040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T00:38:59.495 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:59.494+0000 7fa9b3932040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T00:38:59.565 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T00:38:59.564+0000 7fcaa75fc640 -1 Falling back to public interface 2024-05-27T00:38:59.572 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T00:38:59.571+0000 7f76e320d640 -1 Falling back to public interface 2024-05-27T00:38:59.573 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T00:38:59.572+0000 7f6768213640 -1 Falling back to public interface 2024-05-27T00:38:59.588 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T00:38:59.587+0000 7f3534e86640 -1 Falling back to public interface 2024-05-27T00:38:59.598 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T00:38:59.597+0000 7f62bce3d640 -1 Falling back to public interface 2024-05-27T00:38:59.599 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T00:38:59.598+0000 7fb694353640 -1 Falling back to public interface 2024-05-27T00:38:59.600 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T00:38:59.598+0000 7f2e83213640 -1 Falling back to public interface 2024-05-27T00:38:59.600 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T00:38:59.599+0000 7f2e81b3d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-05-27T00:38:59.603 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T00:38:59.602+0000 7f230b030640 -1 Falling back to public interface 2024-05-27T00:38:59.665 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.664+0000 7f5fb4960040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T00:38:59.804 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:59.803+0000 7fa9b3932040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T00:38:59.853 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:38:59.852+0000 7fa9b3932040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T00:38:59.921 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.920+0000 7f5fb4960040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T00:38:59.972 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:38:59.971+0000 7f5fb4960040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T00:38:59.995 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:00.013 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.012+0000 7fa9b3932040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T00:39:00.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.085+0000 7fd7e8a12640 1 Processor -- start 2024-05-27T00:39:00.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.085+0000 7fd7e8a12640 1 -- start start 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e8a12640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7e40b5270 0x7fd7e40d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e8a12640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e40b3aa0 con 0x7fd7e40d1240 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e8a12640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e40d2ac0 con 0x7fd7e4161a60 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e8a12640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e40d2da0 con 0x7fd7e40b5270 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7e40b5270 0x7fd7e40d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e1d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd7e40d1240 0x7fd7e40ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39932/0 (socket says 172.21.15.111:39932) 2024-05-27T00:39:00.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e1d9b640 1 -- 172.21.15.111:0/3941098701 learned_addr learned my addr 172.21.15.111:0/3941098701 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:00.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7e40b5270 0x7fd7e40d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49248/0 (socket says 172.21.15.111:49248) 2024-05-27T00:39:00.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e2d9d640 1 -- 172.21.15.111:0/3941098701 >> v1:172.21.15.153:6789/0 conn(0x7fd7e4161a60 legacy=0x7fd7e40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:00.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e2d9d640 1 -- 172.21.15.111:0/3941098701 >> v1:172.21.15.111:6789/0 conn(0x7fd7e40d1240 legacy=0x7fd7e40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:00.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e2d9d640 1 -- 172.21.15.111:0/3941098701 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e40d3080 con 0x7fd7e40b5270 2024-05-27T00:39:00.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.086+0000 7fd7e159a640 1 -- 172.21.15.111:0/3941098701 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 995557573 0 0) 0x7fd7e40b3aa0 con 0x7fd7e40d1240 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.087+0000 7fd7e2d9d640 1 --2- 172.21.15.111:0/3941098701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e40d24e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009d90 tx=0x7fd7dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b02096759670bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.087+0000 7fd7e159a640 1 -- 172.21.15.111:0/3941098701 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc007740 con 0x7fd7e40b5270 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.087+0000 7fd7e159a640 1 -- 172.21.15.111:0/3941098701 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7dc0078e0 con 0x7fd7e40b5270 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.087+0000 7fd7e159a640 1 -- 172.21.15.111:0/3941098701 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc007cb0 con 0x7fd7e40b5270 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.087+0000 7fd7e8a12640 1 -- 172.21.15.111:0/3941098701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 msgr2=0x7fd7e40d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.087+0000 7fd7e8a12640 1 --2- 172.21.15.111:0/3941098701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e40d24e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009d90 tx=0x7fd7dc02a6f0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.090 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.088+0000 7fd7e8a12640 1 -- 172.21.15.111:0/3941098701 shutdown_connections 2024-05-27T00:39:00.091 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.088+0000 7fd7e8a12640 1 --2- 172.21.15.111:0/3941098701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e40d24e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.091 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.088+0000 7fd7e8a12640 1 -- 172.21.15.111:0/3941098701 >> 172.21.15.111:0/3941098701 conn(0x7fd7e409d470 msgr2=0x7fd7e40d4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:00.091 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.088+0000 7fd7e8a12640 1 -- 172.21.15.111:0/3941098701 shutdown_connections 2024-05-27T00:39:00.091 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.088+0000 7fd7e8a12640 1 -- 172.21.15.111:0/3941098701 wait complete. 2024-05-27T00:39:00.091 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.089+0000 7fd7e8a12640 1 Processor -- start 2024-05-27T00:39:00.091 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.089+0000 7fd7e8a12640 1 -- start start 2024-05-27T00:39:00.093 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e41571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e40d1240 0x7fd7e40b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 0x7fd7e40b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4161e40 con 0x7fd7e40b5270 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e4169200 con 0x7fd7e40d1240 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e40b7180 con 0x7fd7e4161a60 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e41571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e41571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49250/0 (socket says 172.21.15.111:49250) 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 -- 172.21.15.111:0/1083869267 learned_addr learned my addr 172.21.15.111:0/1083869267 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e2d9d640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 0x7fd7e40b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:00.094 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e1d9b640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e40d1240 0x7fd7e40b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e2d9d640 1 -- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 msgr2=0x7fd7e40b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e2d9d640 1 -- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 msgr2=0x7fd7e40b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e2d9d640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 0x7fd7e40b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e2d9d640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 0x7fd7e40b9630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 -- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 msgr2=0x7fd7e40b9630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 0x7fd7e40b9630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 -- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e40d1240 msgr2=0x7fd7e40b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e40d1240 0x7fd7e40b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 -- 172.21.15.111:0/1083869267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e40ce660 con 0x7fd7e40b5270 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e259c640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e41571a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400c400 tx=0x7fd7d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4017070 con 0x7fd7e40b5270 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d40052d0 con 0x7fd7e40b5270 2024-05-27T00:39:00.095 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e40ce0a0 con 0x7fd7e40b5270 2024-05-27T00:39:00.096 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4003830 con 0x7fd7e40b5270 2024-05-27T00:39:00.096 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.090+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e40b2aa0 con 0x7fd7e40b5270 2024-05-27T00:39:00.096 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.091+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7d401a070 con 0x7fd7e40b5270 2024-05-27T00:39:00.096 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.091+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd7d4011b30 con 0x7fd7e40b5270 2024-05-27T00:39:00.096 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.091+0000 7f5fb4960040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T00:39:00.096 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.091+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e4068d90 con 0x7fd7e40b5270 2024-05-27T00:39:00.096 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.093+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd7d4021020 con 0x7fd7e40b5270 2024-05-27T00:39:00.150 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.149+0000 7fa9b3932040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T00:39:00.155 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.154+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7e4074280 con 0x7fd7e40b5270 2024-05-27T00:39:00.156 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.155+0000 7fd7d2ffd640 1 -- 172.21.15.111:0/1083869267 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd7d4011de0 con 0x7fd7e40b5270 2024-05-27T00:39:00.156 INFO:teuthology.misc.health.smithi111.stdout: 2024-05-27T00:39:00.156 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":10,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:38:58.939697+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:00.157 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 msgr2=0x7fd7e41571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:00.157 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e41571a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400c400 tx=0x7fd7d400c8c0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.157 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 shutdown_connections 2024-05-27T00:39:00.157 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd7e4161a60 0x7fd7e40b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.157 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e40d1240 0x7fd7e40b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.157 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 --2- 172.21.15.111:0/1083869267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7e40b5270 0x7fd7e41571a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:00.158 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 >> 172.21.15.111:0/1083869267 conn(0x7fd7e409d470 msgr2=0x7fd7e406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:00.158 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 shutdown_connections 2024-05-27T00:39:00.158 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:00.156+0000 7fd7e8a12640 1 -- 172.21.15.111:0/1083869267 wait complete. 2024-05-27T00:39:00.166 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-27T00:39:00.200 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.199+0000 7fa9b3932040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T00:39:00.270 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.269+0000 7fa9b3932040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T00:39:00.315 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.314+0000 7fa9b3932040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T00:39:00.390 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.389+0000 7f5fb4960040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T00:39:00.439 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.438+0000 7f5fb4960040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T00:39:00.592 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.591+0000 7f5fb4960040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T00:39:00.729 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.728+0000 7f5fb4960040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T00:39:00.777 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.776+0000 7f5fb4960040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T00:39:00.794 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.793+0000 7fa9b3932040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T00:39:00.846 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.845+0000 7f5fb4960040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T00:39:00.890 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:00.889+0000 7f5fb4960040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T00:39:00.911 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:00.910+0000 7fa9b3932040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T00:39:01.366 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:01.365+0000 7f5fb4960040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T00:39:01.483 INFO:tasks.ceph.mgr.y.smithi111.stderr:2024-05-27T00:39:01.480+0000 7f5fb4960040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T00:39:01.926 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:01.924+0000 7fa9542bc640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-27T00:39:01.926 INFO:tasks.ceph.mgr.x.smithi153.stderr:2024-05-27T00:39:01.925+0000 7fa9542bc640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-27T00:39:03.043 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T00:39:03.042+0000 7fcaa75fc640 -1 osd.1 0 log_to_monitors true 2024-05-27T00:39:03.049 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T00:39:03.047+0000 7f6768213640 -1 osd.4 0 log_to_monitors true 2024-05-27T00:39:03.049 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T00:39:03.047+0000 7f76e320d640 -1 osd.5 0 log_to_monitors true 2024-05-27T00:39:03.079 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T00:39:03.078+0000 7f3534e86640 -1 osd.2 0 log_to_monitors true 2024-05-27T00:39:03.094 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T00:39:03.092+0000 7f62bce3d640 -1 osd.7 0 log_to_monitors true 2024-05-27T00:39:03.120 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T00:39:03.119+0000 7f230b030640 -1 osd.6 0 log_to_monitors true 2024-05-27T00:39:03.126 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T00:39:03.125+0000 7f2e83213640 -1 osd.3 0 log_to_monitors true 2024-05-27T00:39:03.126 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T00:39:03.126+0000 7fb694353640 -1 osd.0 0 log_to_monitors true 2024-05-27T00:39:06.963 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T00:39:06.962+0000 7fcaa3592640 -1 osd.1 0 waiting for initial osdmap 2024-05-27T00:39:06.964 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T00:39:06.961+0000 7f6763af3640 -1 osd.4 0 waiting for initial osdmap 2024-05-27T00:39:06.970 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T00:39:06.969+0000 7f62b8dd3640 -1 osd.7 0 waiting for initial osdmap 2024-05-27T00:39:06.973 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:06.973 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T00:39:06.972+0000 7fca9b3b2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:06.977 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T00:39:06.976+0000 7f675b913640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:06.988 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T00:39:06.987+0000 7f62b0bf3640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:07.005 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T00:39:07.004+0000 7fb6902e9640 -1 osd.0 0 waiting for initial osdmap 2024-05-27T00:39:07.020 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T00:39:07.017+0000 7f2e7eb24640 -1 osd.3 0 waiting for initial osdmap 2024-05-27T00:39:07.020 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T00:39:07.019+0000 7fb688109640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:07.032 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T00:39:07.032+0000 7f3531630640 -1 osd.2 0 waiting for initial osdmap 2024-05-27T00:39:07.037 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T00:39:07.037+0000 7f2e76944640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:07.044 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T00:39:07.044+0000 7f3528c3c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:07.078 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.077+0000 7faf50638640 1 Processor -- start 2024-05-27T00:39:07.078 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.077+0000 7faf50638640 1 -- start start 2024-05-27T00:39:07.080 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf50638640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4815e480 0x7faf480b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:07.080 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf50638640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf4806d840 0x7faf4806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:07.080 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf50638640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf480b3d10 con 0x7faf48072530 2024-05-27T00:39:07.080 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf50638640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7faf480b4920 con 0x7faf4806d840 2024-05-27T00:39:07.080 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf50638640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faf480b3ff0 con 0x7faf4815e480 2024-05-27T00:39:07.080 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4ebd5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faf4806d840 0x7faf4806dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4dbd3640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7faf48072530 0x7faf480b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:48616/0 (socket says 172.21.15.111:48616) 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4dbd3640 1 -- 172.21.15.111:0/3188962047 learned_addr learned my addr 172.21.15.111:0/3188962047 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4e3d4640 1 --2- 172.21.15.111:0/3188962047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4815e480 0x7faf480b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4ebd5640 1 -- 172.21.15.111:0/3188962047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4815e480 msgr2=0x7faf480b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4ebd5640 1 --2- 172.21.15.111:0/3188962047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4815e480 0x7faf480b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4ebd5640 1 -- 172.21.15.111:0/3188962047 >> v1:172.21.15.111:6789/0 conn(0x7faf48072530 legacy=0x7faf480b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4d3d2640 1 -- 172.21.15.111:0/3188962047 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846590207 0 0) 0x7faf480b3d10 con 0x7faf48072530 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.078+0000 7faf4ebd5640 1 -- 172.21.15.111:0/3188962047 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf480b42d0 con 0x7faf4806d840 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.079+0000 7faf4ebd5640 1 --2- 172.21.15.111:0/3188962047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4806d840 0x7faf4806dca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf40009d90 tx=0x7faf4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b59c14755f5c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.079+0000 7faf4d3d2640 1 -- 172.21.15.111:0/3188962047 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40007740 con 0x7faf4806d840 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.079+0000 7faf4d3d2640 1 -- 172.21.15.111:0/3188962047 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf400078e0 con 0x7faf4806d840 2024-05-27T00:39:07.081 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.079+0000 7faf4d3d2640 1 -- 172.21.15.111:0/3188962047 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40007cb0 con 0x7faf4806d840 2024-05-27T00:39:07.085 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 -- 172.21.15.111:0/3188962047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4806d840 msgr2=0x7faf4806dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 --2- 172.21.15.111:0/3188962047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4806d840 0x7faf4806dca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf40009d90 tx=0x7faf4002a660 comp rx=0 tx=0).stop 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 -- 172.21.15.111:0/3188962047 shutdown_connections 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 --2- 172.21.15.111:0/3188962047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf4806d840 0x7faf4806dca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 --2- 172.21.15.111:0/3188962047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4815e480 0x7faf480b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 -- 172.21.15.111:0/3188962047 >> 172.21.15.111:0/3188962047 conn(0x7faf48071910 msgr2=0x7faf48071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 -- 172.21.15.111:0/3188962047 shutdown_connections 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.080+0000 7faf50638640 1 -- 172.21.15.111:0/3188962047 wait complete. 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.081+0000 7faf50638640 1 Processor -- start 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.081+0000 7faf50638640 1 -- start start 2024-05-27T00:39:07.086 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4806d840 0x7faf480cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 0x7faf48150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf4815e480 0x7faf4815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf480b54c0 con 0x7faf48072530 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faf48072970 con 0x7faf4815e480 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faf4806c2e0 con 0x7faf4806d840 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 0x7faf48150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 0x7faf48150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38626/0 (socket says 172.21.15.111:38626) 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 -- 172.21.15.111:0/1411085437 learned_addr learned my addr 172.21.15.111:0/1411085437 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4e3d4640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4806d840 0x7faf480cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4ebd5640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf4815e480 0x7faf4815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 -- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4806d840 msgr2=0x7faf480cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4806d840 0x7faf480cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 -- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf4815e480 msgr2=0x7faf4815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:07.087 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf4815e480 0x7faf4815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 -- 172.21.15.111:0/1411085437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf480cde80 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4ebd5640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf4815e480 0x7faf4815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf4dbd3640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 0x7faf48150a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c4a0 tx=0x7faf3800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38018070 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf380074e0 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38004a20 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf480ce160 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.082+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf480ce720 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.083+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf48068d90 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.084+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7faf3801b070 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.084+0000 7faf3effd640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faf2c06bdd0 0x7faf2c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.084+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7faf38089160 con 0x7faf48072530 2024-05-27T00:39:07.088 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.084+0000 7faf4e3d4640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faf2c06bdd0 0x7faf2c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:07.089 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.085+0000 7faf4e3d4640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faf2c06bdd0 0x7faf2c06e290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf44009aa0 tx=0x7faf44009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:07.089 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.089+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faf380583c0 con 0x7faf48072530 2024-05-27T00:39:07.178 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T00:39:07.176+0000 7f2306fc6640 -1 osd.6 0 waiting for initial osdmap 2024-05-27T00:39:07.178 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T00:39:07.176+0000 7f76df3ac640 -1 osd.5 0 waiting for initial osdmap 2024-05-27T00:39:07.192 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T00:39:07.191+0000 7f22fede6640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:07.193 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T00:39:07.192+0000 7f76d69b8640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T00:39:07.266 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.265+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf48062380 con 0x7faf48072530 2024-05-27T00:39:07.267 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.266+0000 7faf3effd640 1 -- 172.21.15.111:0/1411085437 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7faf3805c6c0 con 0x7faf48072530 2024-05-27T00:39:07.267 INFO:teuthology.misc.health.smithi111.stdout: 2024-05-27T00:39:07.267 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":12,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:04.926045+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:07.269 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faf2c06bdd0 msgr2=0x7faf2c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:07.269 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faf2c06bdd0 0x7faf2c06e290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf44009aa0 tx=0x7faf44009190 comp rx=0 tx=0).stop 2024-05-27T00:39:07.269 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 msgr2=0x7faf48150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:07.269 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 0x7faf48150a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c4a0 tx=0x7faf3800c960 comp rx=0 tx=0).stop 2024-05-27T00:39:07.269 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 shutdown_connections 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faf2c06bdd0 0x7faf2c06e290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf4815e480 0x7faf4815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf48072530 0x7faf48150a80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 --2- 172.21.15.111:0/1411085437 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faf4806d840 0x7faf480cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 >> 172.21.15.111:0/1411085437 conn(0x7faf48071910 msgr2=0x7faf4806dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 shutdown_connections 2024-05-27T00:39:07.270 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:07.268+0000 7faf50638640 1 -- 172.21.15.111:0/1411085437 wait complete. 2024-05-27T00:39:07.278 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-27T00:39:14.085 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:14.187 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.186+0000 7f94d284d640 1 Processor -- start 2024-05-27T00:39:14.187 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.186+0000 7f94d284d640 1 -- start start 2024-05-27T00:39:14.188 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94d284d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94cc0d1240 0x7f94cc0d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.188 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94d284d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94cc0b3aa0 con 0x7f94cc161a60 2024-05-27T00:39:14.188 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94d284d640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94cc0d2ac0 con 0x7f94cc0b5270 2024-05-27T00:39:14.188 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94d284d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f94cc0d2da0 con 0x7f94cc0d1240 2024-05-27T00:39:14.188 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94cb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f94cc0d1240 0x7f94cc0d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.189 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94cbfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f94cc161a60 0x7f94cc0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43352/0 (socket says 172.21.15.111:43352) 2024-05-27T00:39:14.189 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94cbfff640 1 -- 172.21.15.111:0/2804900855 learned_addr learned my addr 172.21.15.111:0/2804900855 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:14.189 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94cb7fe640 1 -- 172.21.15.111:0/2804900855 >> v1:172.21.15.153:6789/0 conn(0x7f94cc0b5270 legacy=0x7f94cc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.189 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94cb7fe640 1 -- 172.21.15.111:0/2804900855 >> v1:172.21.15.111:6789/0 conn(0x7f94cc161a60 legacy=0x7f94cc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.189 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94caffd640 1 -- 172.21.15.111:0/2804900855 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1841524698 0 0) 0x7f94cc0b3aa0 con 0x7f94cc161a60 2024-05-27T00:39:14.189 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.187+0000 7f94cb7fe640 1 -- 172.21.15.111:0/2804900855 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc0d3080 con 0x7f94cc0d1240 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.188+0000 7f94cb7fe640 1 --2- 172.21.15.111:0/2804900855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc0d1240 0x7f94cc0d24e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94c00097a0 tx=0x7f94c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1544b16c4f1967 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.188+0000 7f94caffd640 1 -- 172.21.15.111:0/2804900855 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0037070 con 0x7f94cc0d1240 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.188+0000 7f94caffd640 1 -- 172.21.15.111:0/2804900855 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c0032440 con 0x7f94cc0d1240 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.188+0000 7f94caffd640 1 -- 172.21.15.111:0/2804900855 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0032810 con 0x7f94cc0d1240 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.188+0000 7f94d284d640 1 -- 172.21.15.111:0/2804900855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc0d1240 msgr2=0x7f94cc0d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.188+0000 7f94d284d640 1 --2- 172.21.15.111:0/2804900855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc0d1240 0x7f94cc0d24e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94c00097a0 tx=0x7f94c002a1e0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.190 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.189+0000 7f94d284d640 1 -- 172.21.15.111:0/2804900855 shutdown_connections 2024-05-27T00:39:14.191 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.189+0000 7f94d284d640 1 --2- 172.21.15.111:0/2804900855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc0d1240 0x7f94cc0d24e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.191 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.189+0000 7f94d284d640 1 -- 172.21.15.111:0/2804900855 >> 172.21.15.111:0/2804900855 conn(0x7f94cc09d470 msgr2=0x7f94cc0d4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:14.191 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.189+0000 7f94d284d640 1 -- 172.21.15.111:0/2804900855 shutdown_connections 2024-05-27T00:39:14.191 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.189+0000 7f94d284d640 1 -- 172.21.15.111:0/2804900855 wait complete. 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.190+0000 7f94d284d640 1 Processor -- start 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.190+0000 7f94d284d640 1 -- start start 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b5270 0x7f94cc0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc0d1240 0x7f94cc0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 0x7f94cc1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc161e40 con 0x7f94cc161a60 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc169200 con 0x7f94cc0d1240 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc0c9260 con 0x7f94cc0b5270 2024-05-27T00:39:14.193 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 0x7f94cc1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 0x7f94cc1504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47776/0 (socket says 172.21.15.111:47776) 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 -- 172.21.15.111:0/589785157 learned_addr learned my addr 172.21.15.111:0/589785157 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94cb7fe640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc0d1240 0x7f94cc0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94cbfff640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b5270 0x7f94cc0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 -- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b5270 msgr2=0x7f94cc0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b5270 0x7f94cc0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 -- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc0d1240 msgr2=0x7f94cc0cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc0d1240 0x7f94cc0cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 -- 172.21.15.111:0/589785157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc0c9540 con 0x7f94cc161a60 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94cb7fe640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc0d1240 0x7f94cc0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d0dea640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 0x7f94cc1504c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c780 tx=0x7f94c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.194 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4004b60 con 0x7f94cc161a60 2024-05-27T00:39:14.195 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c4004d00 con 0x7f94cc161a60 2024-05-27T00:39:14.195 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400aa90 con 0x7f94cc161a60 2024-05-27T00:39:14.195 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc0c9820 con 0x7f94cc161a60 2024-05-27T00:39:14.195 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.191+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc0c9de0 con 0x7f94cc161a60 2024-05-27T00:39:14.195 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.193+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc068d90 con 0x7f94cc161a60 2024-05-27T00:39:14.196 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.193+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94c400ac30 con 0x7f94cc161a60 2024-05-27T00:39:14.196 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.193+0000 7f94c8ff9640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f94b806be30 0x7f94b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.196 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.194+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f94c408ae60 con 0x7f94cc161a60 2024-05-27T00:39:14.196 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.194+0000 7f94cbfff640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f94b806be30 0x7f94b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.196 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.194+0000 7f94cbfff640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f94b806be30 0x7f94b806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94bc000f60 tx=0x7f94bc007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.199 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.198+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f94c4020070 con 0x7f94cc161a60 2024-05-27T00:39:14.373 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.372+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94cc0cff50 con 0x7f94cc161a60 2024-05-27T00:39:14.374 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.373+0000 7f94c8ff9640 1 -- 172.21.15.111:0/589785157 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f94c4059ad0 con 0x7f94cc161a60 2024-05-27T00:39:14.374 INFO:teuthology.misc.health.smithi111.stdout: 2024-05-27T00:39:14.374 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":16,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:11.191183+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:14.376 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f94b806be30 msgr2=0x7f94b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.376 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f94b806be30 0x7f94b806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94bc000f60 tx=0x7f94bc007c00 comp rx=0 tx=0).stop 2024-05-27T00:39:14.376 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 msgr2=0x7f94cc1504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.376 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 0x7f94cc1504c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c780 tx=0x7f94c400cc40 comp rx=0 tx=0).stop 2024-05-27T00:39:14.376 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 shutdown_connections 2024-05-27T00:39:14.376 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f94b806be30 0x7f94b806e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.377 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94cc161a60 0x7f94cc1504c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.377 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94cc0d1240 0x7f94cc0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.377 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 --2- 172.21.15.111:0/589785157 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f94cc0b5270 0x7f94cc0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.377 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 >> 172.21.15.111:0/589785157 conn(0x7f94cc09d470 msgr2=0x7f94cc0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:14.377 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 shutdown_connections 2024-05-27T00:39:14.377 INFO:teuthology.misc.health.smithi111.stderr:2024-05-27T00:39:14.375+0000 7f94d284d640 1 -- 172.21.15.111:0/589785157 wait complete. 2024-05-27T00:39:14.386 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-27T00:39:14.386 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-27T00:39:14.386 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-27T00:39:14.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-27T00:39:14.389 INFO:tasks.ceph.mds.a:Started 2024-05-27T00:39:14.389 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-27T00:39:14.389 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-27T00:39:14.391 INFO:tasks.ceph.mds.c:Started 2024-05-27T00:39:14.391 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-27T00:39:14.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-27T00:39:14.394 INFO:tasks.ceph.mds.b:Started 2024-05-27T00:39:14.394 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-27T00:39:14.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-27T00:39:14.396 INFO:tasks.ceph.mds.d:Started 2024-05-27T00:39:14.396 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-27T00:39:14.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:14.463 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2024-05-27T00:39:14.463 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2024-05-27T00:39:14.480 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2024-05-27T00:39:14.481 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2024-05-27T00:39:14.481 INFO:tasks.ceph.mds.b.smithi153.stderr:2024-05-27T00:39:14.480+0000 7f04975a2640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-27T00:39:14.481 INFO:tasks.ceph.mds.b.smithi153.stderr:2024-05-27T00:39:14.480+0000 7f04975a2640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-05-27T00:39:14.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.501+0000 7fa1a006d640 1 Processor -- start 2024-05-27T00:39:14.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.501+0000 7fa1a006d640 1 -- start start 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa1a006d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 0x7fa1980b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa1a006d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa198162560 0x7fa1980d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa1a006d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa19806cc70 con 0x7fa1980b77e0 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa1a006d640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa1980d1e50 con 0x7fa198162560 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa1a006d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa1980d2130 con 0x7fa1980b3630 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa19d608640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa1980b77e0 0x7fa198161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43370/0 (socket says 172.21.15.111:43370) 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa19d608640 1 -- 172.21.15.111:0/2302370206 learned_addr learned my addr 172.21.15.111:0/2302370206 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa19e60a640 1 --2- 172.21.15.111:0/2302370206 >> v2:172.21.15.153:3300/0 conn(0x7fa198162560 0x7fa1980d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa19de09640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 0x7fa1980b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928163467 0 0) 0x7fa19806cc70 con 0x7fa1980b77e0 2024-05-27T00:39:14.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.502+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa184003520 con 0x7fa1980b77e0 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1807211164 0 0) 0x7fa184003520 con 0x7fa1980b77e0 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 msgr2=0x7fa1980b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 0x7fa1980b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 >> v2:172.21.15.153:3300/0 conn(0x7fa198162560 msgr2=0x7fa1980d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 --2- 172.21.15.111:0/2302370206 >> v2:172.21.15.153:3300/0 conn(0x7fa198162560 0x7fa1980d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1980d2410 con 0x7fa1980b77e0 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fa188001670 con 0x7fa1980b77e0 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 >> v1:172.21.15.111:6789/0 conn(0x7fa1980b77e0 legacy=0x7fa198161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1840039b0 0x7fa184003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184004800 0x7fa184004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 0x7fa1840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980d1e50 con 0x7fa184004800 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980d2130 con 0x7fa1840039b0 2024-05-27T00:39:14.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa18400b9c0 con 0x7fa1980b3630 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19d608640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184004800 0x7fa184004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19de09640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1840039b0 0x7fa184003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19e60a640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 0x7fa1840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19d608640 1 -- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 msgr2=0x7fa1840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19d608640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1980b3630 0x7fa1840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19d608640 1 -- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1840039b0 msgr2=0x7fa184003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19d608640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1840039b0 0x7fa184003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19d608640 1 -- 172.21.15.111:0/2302370206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa184003520 con 0x7fa184004800 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.503+0000 7fa19de09640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1840039b0 0x7fa184003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa19d608640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184004800 0x7fa184004c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa188001050 tx=0x7fa184005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977974c6efdffd71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18802e070 con 0x7fa184004800 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa188005af0 con 0x7fa184004800 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa19ce07640 1 -- 172.21.15.111:0/2302370206 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188007d70 con 0x7fa184004800 2024-05-27T00:39:14.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa1a006d640 1 -- 172.21.15.111:0/2302370206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184004800 msgr2=0x7fa184004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2302370206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa184004800 0x7fa184004c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa188001050 tx=0x7fa184005830 comp rx=0 tx=0).stop 2024-05-27T00:39:14.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa19e60a640 1 -- 172.21.15.111:0/2302370206 reap_dead start 2024-05-27T00:39:14.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa1a006d640 1 -- 172.21.15.111:0/2302370206 shutdown_connections 2024-05-27T00:39:14.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.504+0000 7fa1a006d640 1 -- 172.21.15.111:0/2302370206 >> 172.21.15.111:0/2302370206 conn(0x7fa1980d42c0 msgr2=0x7fa1980d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:14.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.505+0000 7fa1a006d640 1 -- 172.21.15.111:0/2302370206 shutdown_connections 2024-05-27T00:39:14.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.505+0000 7fa1a006d640 1 -- 172.21.15.111:0/2302370206 wait complete. 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.506+0000 7fa1a006d640 1 Processor -- start 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.506+0000 7fa1a006d640 1 -- start start 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.506+0000 7fa1a006d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa184004800 0x7fa1980ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 0x7fa19815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1980b5000 0x7fa19814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980b7420 con 0x7fa19815cc50 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980b72a0 con 0x7fa1980b5000 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa1980b75a0 con 0x7fa184004800 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 0x7fa19815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 0x7fa19815b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47818/0 (socket says 172.21.15.111:47818) 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 -- 172.21.15.111:0/2254984667 learned_addr learned my addr 172.21.15.111:0/2254984667 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19e60a640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1980b5000 0x7fa19814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19de09640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa184004800 0x7fa1980ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 -- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa184004800 msgr2=0x7fa1980ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa184004800 0x7fa1980ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 -- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1980b5000 msgr2=0x7fa19814c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1980b5000 0x7fa19814c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 -- 172.21.15.111:0/2254984667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19815bbd0 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa19d608640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 0x7fa19815b330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa188034040 tx=0x7fa188000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188042070 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa188007480 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19815beb0 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188007d50 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.507+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19815c470 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.508+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa198068680 con 0x7fa19815cc50 2024-05-27T00:39:14.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.509+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa188045070 con 0x7fa19815cc50 2024-05-27T00:39:14.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.509+0000 7fa18e7fc640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa17c06be30 0x7fa17c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.509+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa1880ab4c0 con 0x7fa19815cc50 2024-05-27T00:39:14.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.509+0000 7fa19de09640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa17c06be30 0x7fa17c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.510+0000 7fa19de09640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa17c06be30 0x7fa17c06e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa194006f80 tx=0x7fa194008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.514+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa188005ac0 con 0x7fa19815cc50 2024-05-27T00:39:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.691+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1980cef10 con 0x7fa19815cc50 2024-05-27T00:39:14.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.691+0000 7fa18e7fc640 1 -- 172.21.15.111:0/2254984667 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7fa188079ca0 con 0x7fa19815cc50 2024-05-27T00:39:14.693 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:14.693 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:11.191183+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa17c06be30 msgr2=0x7fa17c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa17c06be30 0x7fa17c06e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa194006f80 tx=0x7fa194008040 comp rx=0 tx=0).stop 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 msgr2=0x7fa19815b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 0x7fa19815b330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa188034040 tx=0x7fa188000fe0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 shutdown_connections 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa17c06be30 0x7fa17c06e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1980b5000 0x7fa19814c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa19815cc50 0x7fa19815b330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 --2- 172.21.15.111:0/2254984667 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa184004800 0x7fa1980ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.693+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 >> 172.21.15.111:0/2254984667 conn(0x7fa1980d42c0 msgr2=0x7fa1980cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.694+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 shutdown_connections 2024-05-27T00:39:14.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.694+0000 7fa1a006d640 1 -- 172.21.15.111:0/2254984667 wait complete. 2024-05-27T00:39:14.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T00:39:14.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:14.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 Processor -- start 2024-05-27T00:39:14.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 -- start start 2024-05-27T00:39:14.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f22800ce140 0x7f22800cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22800b4fd0 0x7f228015f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f228015f5f0 0x7f2280161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f22800b5460 con 0x7f22800b4fd0 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f22800d0200 con 0x7f22800ce140 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2286412640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f22800d04e0 con 0x7f228015f5f0 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.807+0000 7f2277fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22800b4fd0 0x7f228015f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22800b4fd0 0x7f228015f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47824/0 (socket says 172.21.15.111:47824) 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 -- 172.21.15.111:0/1690917523 learned_addr learned my addr 172.21.15.111:0/1690917523 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 -- 172.21.15.111:0/1690917523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f228015f5f0 msgr2=0x7f2280161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f22849af640 1 --2- 172.21.15.111:0/1690917523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f228015f5f0 0x7f2280161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f227ffff640 1 --2- 172.21.15.111:0/1690917523 >> v2:172.21.15.153:3300/0 conn(0x7f22800ce140 0x7f22800cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 --2- 172.21.15.111:0/1690917523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f228015f5f0 0x7f2280161ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 -- 172.21.15.111:0/1690917523 >> v2:172.21.15.153:3300/0 conn(0x7f22800ce140 msgr2=0x7f22800cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:14.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 --2- 172.21.15.111:0/1690917523 >> v2:172.21.15.153:3300/0 conn(0x7f22800ce140 0x7f22800cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 -- 172.21.15.111:0/1690917523 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22800d07c0 con 0x7f22800b4fd0 2024-05-27T00:39:14.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.808+0000 7f2277fff640 1 --2- 172.21.15.111:0/1690917523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800b4fd0 0x7f228015f0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22800d2bb0 tx=0x7f226802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749c09968e0488dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f227f7fe640 1 -- 172.21.15.111:0/1690917523 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268037070 con 0x7f22800b4fd0 2024-05-27T00:39:14.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f227f7fe640 1 -- 172.21.15.111:0/1690917523 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226802ae00 con 0x7f22800b4fd0 2024-05-27T00:39:14.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f227f7fe640 1 -- 172.21.15.111:0/1690917523 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268032660 con 0x7f22800b4fd0 2024-05-27T00:39:14.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 -- 172.21.15.111:0/1690917523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800b4fd0 msgr2=0x7f228015f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 --2- 172.21.15.111:0/1690917523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800b4fd0 0x7f228015f0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22800d2bb0 tx=0x7f226802a0b0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 -- 172.21.15.111:0/1690917523 shutdown_connections 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 --2- 172.21.15.111:0/1690917523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f228015f5f0 0x7f2280161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 --2- 172.21.15.111:0/1690917523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800b4fd0 0x7f228015f0b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 --2- 172.21.15.111:0/1690917523 >> v2:172.21.15.153:3300/0 conn(0x7f22800ce140 0x7f22800cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 -- 172.21.15.111:0/1690917523 >> 172.21.15.111:0/1690917523 conn(0x7f228006ba70 msgr2=0x7f22800d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 -- 172.21.15.111:0/1690917523 shutdown_connections 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.809+0000 7f2286412640 1 -- 172.21.15.111:0/1690917523 wait complete. 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.810+0000 7f2286412640 1 Processor -- start 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.810+0000 7f2286412640 1 -- start start 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2286412640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22800b4fd0 0x7f22800c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2286412640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 0x7f22800c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2286412640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2280152710 0x7f22801e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2286412640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2280162580 con 0x7f22800ce140 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2286412640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2280162400 con 0x7f22800b4fd0 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2286412640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2280162700 con 0x7f2280152710 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 0x7f22800c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 0x7f22800c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47832/0 (socket says 172.21.15.111:47832) 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 -- 172.21.15.111:0/1084106743 learned_addr learned my addr 172.21.15.111:0/1084106743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 -- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2280152710 msgr2=0x7f22801e9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2280152710 0x7f22801e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 -- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22800b4fd0 msgr2=0x7f22800c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22800b4fd0 0x7f22800c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.811+0000 7f2277fff640 1 -- 172.21.15.111:0/1084106743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22800c6730 con 0x7f22800ce140 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.812+0000 7f2277fff640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 0x7f22800c5fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2268000c00 tx=0x7f22680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.812+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268041030 con 0x7f22800ce140 2024-05-27T00:39:14.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.812+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22801541a0 con 0x7f22800ce140 2024-05-27T00:39:14.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.812+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2280154760 con 0x7f22800ce140 2024-05-27T00:39:14.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.812+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226802acc0 con 0x7f22800ce140 2024-05-27T00:39:14.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.812+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268037040 con 0x7f22800ce140 2024-05-27T00:39:14.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.813+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2264005290 con 0x7f22800ce140 2024-05-27T00:39:14.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.813+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2268044070 con 0x7f22800ce140 2024-05-27T00:39:14.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.814+0000 7f227d7fa640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f226006be30 0x7f226006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:14.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.814+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f22680aae10 con 0x7f22800ce140 2024-05-27T00:39:14.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.814+0000 7f227ffff640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f226006be30 0x7f226006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:14.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.815+0000 7f227ffff640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f226006be30 0x7f226006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22800c5c40 tx=0x7f2270008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:14.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.818+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f226804b070 con 0x7f22800ce140 2024-05-27T00:39:14.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.993+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2264005850 con 0x7f22800ce140 2024-05-27T00:39:14.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.993+0000 7f227d7fa640 1 -- 172.21.15.111:0/1084106743 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f2268079a90 con 0x7f22800ce140 2024-05-27T00:39:14.995 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f226006be30 msgr2=0x7f226006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f226006be30 0x7f226006e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22800c5c40 tx=0x7f2270008040 comp rx=0 tx=0).stop 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 msgr2=0x7f22800c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 0x7f22800c5fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2268000c00 tx=0x7f22680046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 shutdown_connections 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f226006be30 0x7f226006e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2280152710 0x7f22801e9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22800ce140 0x7f22800c5fe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 --2- 172.21.15.111:0/1084106743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22800b4fd0 0x7f22800c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:14.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 >> 172.21.15.111:0/1084106743 conn(0x7f228006ba70 msgr2=0x7f22801600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:14.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 shutdown_connections 2024-05-27T00:39:14.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:14.995+0000 7f2286412640 1 -- 172.21.15.111:0/1084106743 wait complete. 2024-05-27T00:39:15.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:15.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.108+0000 7ff6a2542640 1 Processor -- start 2024-05-27T00:39:15.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.108+0000 7ff6a2542640 1 -- start start 2024-05-27T00:39:15.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a2542640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff69c13ee40 0x7ff69c15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:15.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a2542640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c15fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:15.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a2542640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69c160760 con 0x7ff69c160130 2024-05-27T00:39:15.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a2542640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff69c141b00 con 0x7ff69c13ee40 2024-05-27T00:39:15.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a2542640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c160a40 con 0x7ff69c13f220 2024-05-27T00:39:15.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff693fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c15fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff69bfff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff69c13ee40 0x7ff69c15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a0adf640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff69c160130 0x7ff69c0b5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:38790/0 (socket says 172.21.15.111:38790) 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff69bfff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff69c13ee40 0x7ff69c15d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56278/0 (socket says 172.21.15.111:56278) 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.109+0000 7ff6a0adf640 1 -- 172.21.15.111:0/3489324641 learned_addr learned my addr 172.21.15.111:0/3489324641 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.110+0000 7ff69bfff640 1 -- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 msgr2=0x7ff69c15fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.110+0000 7ff69bfff640 1 --2- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c15fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.110+0000 7ff69bfff640 1 -- 172.21.15.111:0/3489324641 >> v1:172.21.15.153:6789/0 conn(0x7ff69c160130 legacy=0x7ff69c0b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.110+0000 7ff69bfff640 1 -- 172.21.15.111:0/3489324641 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c15c950 con 0x7ff69c13ee40 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff69bfff640 1 --2- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c15d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0097a0 tx=0x7ff68c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab7fdf7b8255894e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff69b7fe640 1 -- 172.21.15.111:0/3489324641 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c037070 con 0x7ff69c13ee40 2024-05-27T00:39:15.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff69b7fe640 1 -- 172.21.15.111:0/3489324641 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c02ad10 con 0x7ff69c13ee40 2024-05-27T00:39:15.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff69b7fe640 1 -- 172.21.15.111:0/3489324641 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c032660 con 0x7ff69c13ee40 2024-05-27T00:39:15.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 -- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 msgr2=0x7ff69c15d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 --2- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c15d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0097a0 tx=0x7ff68c02a080 comp rx=0 tx=0).stop 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 -- 172.21.15.111:0/3489324641 shutdown_connections 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 --2- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 --2- 172.21.15.111:0/3489324641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c15d150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 -- 172.21.15.111:0/3489324641 >> 172.21.15.111:0/3489324641 conn(0x7ff69c073130 msgr2=0x7ff69c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 -- 172.21.15.111:0/3489324641 shutdown_connections 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.111+0000 7ff6a2542640 1 -- 172.21.15.111:0/3489324641 wait complete. 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.112+0000 7ff6a2542640 1 Processor -- start 2024-05-27T00:39:15.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.112+0000 7ff6a2542640 1 -- start start 2024-05-27T00:39:15.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a2542640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a2542640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c0ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a2542640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 0x7ff69c0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a2542640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c0b5e40 con 0x7ff69c160130 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a2542640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c15c600 con 0x7ff69c13ee40 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a2542640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c0b5fc0 con 0x7ff69c13f220 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a0adf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 0x7ff69c0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a0adf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 0x7ff69c0afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47840/0 (socket says 172.21.15.111:47840) 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.113+0000 7ff6a0adf640 1 -- 172.21.15.111:0/2731156425 learned_addr learned my addr 172.21.15.111:0/2731156425 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff69bfff640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff693fff640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c0ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a0adf640 1 -- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 msgr2=0x7ff69c0ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a0adf640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c0ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a0adf640 1 -- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 msgr2=0x7ff69c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a0adf640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a0adf640 1 -- 172.21.15.111:0/2731156425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c15aad0 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff693fff640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a0adf640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 0x7ff69c0afa70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff69400caf0 tx=0x7ff6940020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694018070 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff69400a6e0 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69400ab10 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c15adb0 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.114+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c0affb0 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.115+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c068680 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.115+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff69400acb0 con 0x7ff69c160130 2024-05-27T00:39:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.116+0000 7ff6997fa640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff67c06be60 0x7ff67c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:15.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.116+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff69408a210 con 0x7ff69c160130 2024-05-27T00:39:15.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.116+0000 7ff69bfff640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff67c06be60 0x7ff67c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:15.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.117+0000 7ff69bfff640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff67c06be60 0x7ff67c06e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff68c034000 tx=0x7ff68c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:15.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.120+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff694058840 con 0x7ff69c160130 2024-05-27T00:39:15.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.330+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff69c06c100 con 0x7ff69c160130 2024-05-27T00:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.331+0000 7ff6997fa640 1 -- 172.21.15.111:0/2731156425 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7ff69405cb40 con 0x7ff69c160130 2024-05-27T00:39:15.332 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:15.332 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T00:39:15.332 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 2 2024-05-27T00:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff67c06be60 msgr2=0x7ff67c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff67c06be60 0x7ff67c06e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff68c034000 tx=0x7ff68c02e000 comp rx=0 tx=0).stop 2024-05-27T00:39:15.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 msgr2=0x7ff69c0afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 0x7ff69c0afa70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff69400caf0 tx=0x7ff6940020b0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 shutdown_connections 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff67c06be60 0x7ff67c06e320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff69c160130 0x7ff69c0afa70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff69c13f220 0x7ff69c0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 --2- 172.21.15.111:0/2731156425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff69c13ee40 0x7ff69c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 >> 172.21.15.111:0/2731156425 conn(0x7ff69c073130 msgr2=0x7ff69c15ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 shutdown_connections 2024-05-27T00:39:15.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:15.333+0000 7ff6a2542640 1 -- 172.21.15.111:0/2731156425 wait complete. 2024-05-27T00:39:15.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:39:15.622 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:39:17.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:17.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.455+0000 7fe003df4640 1 Processor -- start 2024-05-27T00:39:17.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 -- start start 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdffc144000 0x7fdffc0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc06b390 0x7fdffc0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdffc0cdc80 0x7fdffc0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fdffc0b42f0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdffc0b3440 con 0x7fdffc144000 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.456+0000 7fe003df4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdffc0b36e0 con 0x7fdffc06b390 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdffc0cdc80 0x7fdffc0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdffc0cdc80 0x7fdffc0726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47854/0 (socket says 172.21.15.111:47854) 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 -- 172.21.15.111:0/3582180635 learned_addr learned my addr 172.21.15.111:0/3582180635 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe00138f640 1 --2- 172.21.15.111:0/3582180635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc06b390 0x7fdffc0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe001b90640 1 --2- 172.21.15.111:0/3582180635 >> v2:172.21.15.153:3300/0 conn(0x7fdffc144000 0x7fdffc0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 -- 172.21.15.111:0/3582180635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc06b390 msgr2=0x7fdffc0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 --2- 172.21.15.111:0/3582180635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc06b390 0x7fdffc0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 -- 172.21.15.111:0/3582180635 >> v2:172.21.15.153:3300/0 conn(0x7fdffc144000 msgr2=0x7fdffc0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 --2- 172.21.15.111:0/3582180635 >> v2:172.21.15.153:3300/0 conn(0x7fdffc144000 0x7fdffc0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe002391640 1 -- 172.21.15.111:0/3582180635 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc0b39c0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.457+0000 7fe001b90640 1 --2- 172.21.15.111:0/3582180635 >> v2:172.21.15.153:3300/0 conn(0x7fdffc144000 0x7fdffc0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe002391640 1 --2- 172.21.15.111:0/3582180635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc0726d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0726a0 tx=0x7fdff402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cd4031572eec48 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe000b8e640 1 -- 172.21.15.111:0/3582180635 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4036070 con 0x7fdffc0cdc80 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe000b8e640 1 -- 172.21.15.111:0/3582180635 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff4031600 con 0x7fdffc0cdc80 2024-05-27T00:39:17.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe000b8e640 1 -- 172.21.15.111:0/3582180635 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40319d0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 -- 172.21.15.111:0/3582180635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 msgr2=0x7fdffc0726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 --2- 172.21.15.111:0/3582180635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc0726d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0726a0 tx=0x7fdff402a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 -- 172.21.15.111:0/3582180635 shutdown_connections 2024-05-27T00:39:17.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 --2- 172.21.15.111:0/3582180635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc0726d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 --2- 172.21.15.111:0/3582180635 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc06b390 0x7fdffc0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 --2- 172.21.15.111:0/3582180635 >> v2:172.21.15.153:3300/0 conn(0x7fdffc144000 0x7fdffc0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.458+0000 7fe003df4640 1 -- 172.21.15.111:0/3582180635 >> 172.21.15.111:0/3582180635 conn(0x7fdffc0b1c00 msgr2=0x7fdffc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.459+0000 7fe003df4640 1 -- 172.21.15.111:0/3582180635 shutdown_connections 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.459+0000 7fe003df4640 1 -- 172.21.15.111:0/3582180635 wait complete. 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 Processor -- start 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 -- start start 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc06b390 0x7fdffc13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc144000 0x7fdffc0ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc072c10 con 0x7fdffc0cdc80 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc0b3f80 con 0x7fdffc06b390 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.460+0000 7fe003df4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdffc072d90 con 0x7fdffc144000 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47860/0 (socket says 172.21.15.111:47860) 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 -- 172.21.15.111:0/2869793614 learned_addr learned my addr 172.21.15.111:0/2869793614 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe001b90640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc06b390 0x7fdffc13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe002391640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc144000 0x7fdffc0ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 -- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc144000 msgr2=0x7fdffc0ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc144000 0x7fdffc0ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 -- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc06b390 msgr2=0x7fdffc13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc06b390 0x7fdffc13e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 -- 172.21.15.111:0/2869793614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc154510 con 0x7fdffc0cdc80 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe00138f640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc153b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00d800 tx=0x7fdfec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec018030 con 0x7fdffc0cdc80 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfec004020 con 0x7fdffc0cdc80 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc1547f0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0041f0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.461+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc0afba0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.462+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc068680 con 0x7fdffc0cdc80 2024-05-27T00:39:17.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.463+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfec01b070 con 0x7fdffc0cdc80 2024-05-27T00:39:17.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.463+0000 7fdff27fc640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdfe006be30 0x7fdfe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.463+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdfec08ac50 con 0x7fdffc0cdc80 2024-05-27T00:39:17.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.463+0000 7fe001b90640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdfe006be30 0x7fdfe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.464+0000 7fe001b90640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdfe006be30 0x7fdfe006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdff8006fd0 tx=0x7fdff8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:17.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.467+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdfec090020 con 0x7fdffc0cdc80 2024-05-27T00:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.683+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdffc13fea0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.683+0000 7fdff27fc640 1 -- 172.21.15.111:0/2869793614 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7fdfec0599e0 con 0x7fdffc0cdc80 2024-05-27T00:39:17.684 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:17.685 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T00:39:17.685 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 3 2024-05-27T00:39:17.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdfe006be30 msgr2=0x7fdfe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdfe006be30 0x7fdfe006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdff8006fd0 tx=0x7fdff8008040 comp rx=0 tx=0).stop 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 msgr2=0x7fdffc153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc153b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00d800 tx=0x7fdfec00dcc0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 shutdown_connections 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdfe006be30 0x7fdfe006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdffc144000 0x7fdffc0ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdffc0cdc80 0x7fdffc153b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 --2- 172.21.15.111:0/2869793614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdffc06b390 0x7fdffc13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 >> 172.21.15.111:0/2869793614 conn(0x7fdffc0b1c00 msgr2=0x7fdffc141320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 shutdown_connections 2024-05-27T00:39:17.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.685+0000 7fe003df4640 1 -- 172.21.15.111:0/2869793614 wait complete. 2024-05-27T00:39:17.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:17.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.795+0000 7f55162cd640 1 Processor -- start 2024-05-27T00:39:17.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.795+0000 7f55162cd640 1 -- start start 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f55162cd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f55101476b0 0x7f5510161e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f55162cd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5510162370 0x7f5510162820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f55162cd640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551006d1a0 con 0x7f551015e810 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f55162cd640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f55100c78d0 con 0x7f55101476b0 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f55162cd640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f55100c7bb0 con 0x7f5510162370 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f55101476b0 0x7f5510161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f551486a640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f551015e810 0x7f551015ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43382/0 (socket says 172.21.15.111:43382) 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f551486a640 1 -- 172.21.15.111:0/160259287 learned_addr learned my addr 172.21.15.111:0/160259287 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f55101476b0 0x7f5510161e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47862/0 (socket says 172.21.15.111:47862) 2024-05-27T00:39:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f5507fff640 1 --2- 172.21.15.111:0/160259287 >> v2:172.21.15.153:3300/0 conn(0x7f5510162370 0x7f5510162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550ffff640 1 -- 172.21.15.111:0/160259287 >> v2:172.21.15.153:3300/0 conn(0x7f5510162370 msgr2=0x7f5510162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550ffff640 1 --2- 172.21.15.111:0/160259287 >> v2:172.21.15.153:3300/0 conn(0x7f5510162370 0x7f5510162820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550ffff640 1 -- 172.21.15.111:0/160259287 >> v1:172.21.15.111:6789/0 conn(0x7f551015e810 legacy=0x7f551015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550ffff640 1 -- 172.21.15.111:0/160259287 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551006d480 con 0x7f55101476b0 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.796+0000 7f550f7fe640 1 -- 172.21.15.111:0/160259287 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854717743 0 0) 0x7f551006d1a0 con 0x7f551015e810 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.797+0000 7f550ffff640 1 --2- 172.21.15.111:0/160259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55101476b0 0x7f5510161e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5510073380 tx=0x7f550002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540eec62ef485a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.797+0000 7f550f7fe640 1 -- 172.21.15.111:0/160259287 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500037070 con 0x7f55101476b0 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.797+0000 7f550f7fe640 1 -- 172.21.15.111:0/160259287 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5500032440 con 0x7f55101476b0 2024-05-27T00:39:17.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.797+0000 7f550f7fe640 1 -- 172.21.15.111:0/160259287 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550002ae60 con 0x7f55101476b0 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 -- 172.21.15.111:0/160259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55101476b0 msgr2=0x7f5510161e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 --2- 172.21.15.111:0/160259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55101476b0 0x7f5510161e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5510073380 tx=0x7f550002a110 comp rx=0 tx=0).stop 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 -- 172.21.15.111:0/160259287 shutdown_connections 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 --2- 172.21.15.111:0/160259287 >> v2:172.21.15.153:3300/0 conn(0x7f5510162370 0x7f5510162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 --2- 172.21.15.111:0/160259287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55101476b0 0x7f5510161e30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 -- 172.21.15.111:0/160259287 >> 172.21.15.111:0/160259287 conn(0x7f55100d1f60 msgr2=0x7f55100d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 -- 172.21.15.111:0/160259287 shutdown_connections 2024-05-27T00:39:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.798+0000 7f55162cd640 1 -- 172.21.15.111:0/160259287 wait complete. 2024-05-27T00:39:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.799+0000 7f55162cd640 1 Processor -- start 2024-05-27T00:39:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.799+0000 7f55162cd640 1 -- start start 2024-05-27T00:39:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55101476b0 0x7f551013f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 0x7f55100c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5510162370 0x7f55100cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f551006cd30 con 0x7f551015e810 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5510147ee0 con 0x7f5510162370 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f551006ceb0 con 0x7f55101476b0 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 0x7f55100c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 0x7f55100c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47878/0 (socket says 172.21.15.111:47878) 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 -- 172.21.15.111:0/784931284 learned_addr learned my addr 172.21.15.111:0/784931284 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 -- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55101476b0 msgr2=0x7f551013f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55101476b0 0x7f551013f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 -- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5510162370 msgr2=0x7f55100cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5510162370 0x7f55100cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 -- 172.21.15.111:0/784931284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55101f3d30 con 0x7f551015e810 2024-05-27T00:39:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f5507fff640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 0x7f55100c90c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c820 tx=0x7f54f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8015070 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f8004240 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f800ce10 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55100cbe20 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.800+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55101f3ff0 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.801+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5510068680 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.802+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54f8007830 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.802+0000 7f550d7fa640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54f006be30 0x7f54f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.802+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f54f808a910 con 0x7f551015e810 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.802+0000 7f550ffff640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54f006be30 0x7f54f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.803+0000 7f550ffff640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54f006be30 0x7f54f006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5500034000 tx=0x7f550002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:17.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.806+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f54f80596a0 con 0x7f551015e810 2024-05-27T00:39:17.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.982+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55100c7350 con 0x7f551015e810 2024-05-27T00:39:17.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.983+0000 7f550d7fa640 1 -- 172.21.15.111:0/784931284 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f54f8058fa0 con 0x7f551015e810 2024-05-27T00:39:17.984 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:17.984 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":16,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:11.191183+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:17.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54f006be30 msgr2=0x7f54f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54f006be30 0x7f54f006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5500034000 tx=0x7f550002c000 comp rx=0 tx=0).stop 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 msgr2=0x7f55100c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 0x7f55100c90c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c820 tx=0x7f54f800cce0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 shutdown_connections 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54f006be30 0x7f54f006e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5510162370 0x7f55100cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f551015e810 0x7f55100c90c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 --2- 172.21.15.111:0/784931284 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55101476b0 0x7f551013f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 >> 172.21.15.111:0/784931284 conn(0x7f55100d1f60 msgr2=0x7f551015ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 shutdown_connections 2024-05-27T00:39:17.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:17.985+0000 7f55162cd640 1 -- 172.21.15.111:0/784931284 wait complete. 2024-05-27T00:39:17.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T00:39:17.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:18.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.099+0000 7fc677190640 1 Processor -- start 2024-05-27T00:39:18.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.099+0000 7fc677190640 1 -- start start 2024-05-27T00:39:18.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc677190640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc67013ef60 0x7fc6700cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc677190640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc670142650 con 0x7fc67013f340 2024-05-27T00:39:18.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc677190640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6700cebc0 con 0x7fc6700ccfd0 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc677190640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc6700cf210 con 0x7fc67013ef60 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc674f2c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc67013ef60 0x7fc6700cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc66ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc67013f340 0x7fc6700cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43398/0 (socket says 172.21.15.111:43398) 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc674f2c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc67013ef60 0x7fc6700cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47892/0 (socket says 172.21.15.111:47892) 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc66ffff640 1 -- 172.21.15.111:0/3354514817 learned_addr learned my addr 172.21.15.111:0/3354514817 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc674f2c640 1 -- 172.21.15.111:0/3354514817 >> v1:172.21.15.153:6789/0 conn(0x7fc6700ccfd0 legacy=0x7fc670141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc674f2c640 1 -- 172.21.15.111:0/3354514817 >> v1:172.21.15.111:6789/0 conn(0x7fc67013f340 legacy=0x7fc6700cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc66f7fe640 1 -- 172.21.15.111:0/3354514817 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627352579 0 0) 0x7fc670142650 con 0x7fc67013f340 2024-05-27T00:39:18.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.100+0000 7fc674f2c640 1 -- 172.21.15.111:0/3354514817 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6700ceea0 con 0x7fc67013ef60 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.101+0000 7fc674f2c640 1 --2- 172.21.15.111:0/3354514817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67013ef60 0x7fc6700cf6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc660009d90 tx=0x7fc66002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfb161692e49331 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.101+0000 7fc66f7fe640 1 -- 172.21.15.111:0/3354514817 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660007740 con 0x7fc67013ef60 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.101+0000 7fc66f7fe640 1 -- 172.21.15.111:0/3354514817 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6600078e0 con 0x7fc67013ef60 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.101+0000 7fc66f7fe640 1 -- 172.21.15.111:0/3354514817 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660007cb0 con 0x7fc67013ef60 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 -- 172.21.15.111:0/3354514817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67013ef60 msgr2=0x7fc6700cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 --2- 172.21.15.111:0/3354514817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67013ef60 0x7fc6700cf6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc660009d90 tx=0x7fc66002a6f0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 -- 172.21.15.111:0/3354514817 shutdown_connections 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 --2- 172.21.15.111:0/3354514817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc67013ef60 0x7fc6700cf6a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 -- 172.21.15.111:0/3354514817 >> 172.21.15.111:0/3354514817 conn(0x7fc6700b1c00 msgr2=0x7fc6700b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 -- 172.21.15.111:0/3354514817 shutdown_connections 2024-05-27T00:39:18.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.102+0000 7fc677190640 1 -- 172.21.15.111:0/3354514817 wait complete. 2024-05-27T00:39:18.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.103+0000 7fc677190640 1 Processor -- start 2024-05-27T00:39:18.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.103+0000 7fc677190640 1 -- start start 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc677190640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 0x7fc670140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc677190640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67013ef60 0x7fc670143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc677190640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc67013f340 0x7fc6701443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc677190640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc670142030 con 0x7fc6700ccfd0 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc677190640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc6700ce890 con 0x7fc67013f340 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc677190640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc6701421b0 con 0x7fc67013ef60 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 0x7fc670140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 0x7fc670140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47898/0 (socket says 172.21.15.111:47898) 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 -- 172.21.15.111:0/37823136 learned_addr learned my addr 172.21.15.111:0/37823136 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc66ffff640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67013ef60 0x7fc670143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc67572d640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc67013f340 0x7fc6701443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 -- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67013ef60 msgr2=0x7fc670143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67013ef60 0x7fc670143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 -- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc67013f340 msgr2=0x7fc6701443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc67013f340 0x7fc6701443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 -- 172.21.15.111:0/37823136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67015b730 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc66ffff640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67013ef60 0x7fc670143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc67572d640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc67013f340 0x7fc6701443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.104+0000 7fc674f2c640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 0x7fc670140210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc660009eb0 tx=0x7fc660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.105+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660040070 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.105+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc660031d00 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.105+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660045660 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.105+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67015ba10 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.105+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc670142d20 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.106+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc670068680 con 0x7fc6700ccfd0 2024-05-27T00:39:18.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.106+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc660039040 con 0x7fc6700ccfd0 2024-05-27T00:39:18.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.107+0000 7fc66d7fa640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc65006be30 0x7fc65006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.107+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc6600ac620 con 0x7fc6700ccfd0 2024-05-27T00:39:18.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.107+0000 7fc66ffff640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc65006be30 0x7fc65006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.107+0000 7fc66ffff640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc65006be30 0x7fc65006e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6700af2c0 tx=0x7fc658009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.111+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc66007b290 con 0x7fc6700ccfd0 2024-05-27T00:39:18.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.283+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6700b15e0 con 0x7fc6700ccfd0 2024-05-27T00:39:18.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.283+0000 7fc66d7fa640 1 -- 172.21.15.111:0/37823136 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fc66007ab90 con 0x7fc6700ccfd0 2024-05-27T00:39:18.284 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc65006be30 msgr2=0x7fc65006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc65006be30 0x7fc65006e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6700af2c0 tx=0x7fc658009040 comp rx=0 tx=0).stop 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 msgr2=0x7fc670140210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 0x7fc670140210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc660009eb0 tx=0x7fc660004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 shutdown_connections 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc65006be30 0x7fc65006e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc67013f340 0x7fc6701443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc67013ef60 0x7fc670143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 --2- 172.21.15.111:0/37823136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6700ccfd0 0x7fc670140210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 >> 172.21.15.111:0/37823136 conn(0x7fc6700b1c00 msgr2=0x7fc6700cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 shutdown_connections 2024-05-27T00:39:18.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.285+0000 7fc677190640 1 -- 172.21.15.111:0/37823136 wait complete. 2024-05-27T00:39:18.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-27T00:39:18.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.397+0000 7f139e1c4640 1 Processor -- start 2024-05-27T00:39:18.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.397+0000 7f139e1c4640 1 -- start start 2024-05-27T00:39:18.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.397+0000 7f139e1c4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 0x7f13980b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.397+0000 7f139e1c4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139806cc70 con 0x7f13980b77e0 2024-05-27T00:39:18.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.397+0000 7f139e1c4640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13980d1e50 con 0x7f1398162560 2024-05-27T00:39:18.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.397+0000 7f139e1c4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f13980d2130 con 0x7f13980b3630 2024-05-27T00:39:18.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f138ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f13980b77e0 0x7f1398161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43410/0 (socket says 172.21.15.111:43410) 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f138ffff640 1 -- 172.21.15.111:0/380707362 learned_addr learned my addr 172.21.15.111:0/380707362 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f13977fe640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 0x7f13980b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3567749321 0 0) 0x7f139806cc70 con 0x7f13980b77e0 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1384003520 con 0x7f13980b77e0 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4167919116 0 0) 0x7f13980d1e50 con 0x7f1398162560 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f139806cc70 con 0x7f1398162560 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3079276531 0 0) 0x7f1384003520 con 0x7f13980b77e0 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 msgr2=0x7f13980b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 0x7f13980b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 >> v1:172.21.15.153:6789/0 conn(0x7f1398162560 legacy=0x7f13980d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13980d2410 con 0x7f13980b77e0 2024-05-27T00:39:18.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f1380001670 con 0x7f13980b77e0 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 >> v1:172.21.15.111:6789/0 conn(0x7f13980b77e0 legacy=0x7f1398161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1384003b70 0x7f1384003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 0x7f1384004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f13980d2130 con 0x7f1384004d40 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f138400ba70 con 0x7f1384003b70 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f138400bc20 con 0x7f13980b3630 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f1397fff640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f13977fe640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1384003b70 0x7f1384003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.398+0000 7f138ffff640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 0x7f1384004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1397fff640 1 -- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 msgr2=0x7f1384004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1397fff640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980b3630 0x7f1384004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1397fff640 1 -- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1384003b70 msgr2=0x7f1384003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1397fff640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1384003b70 0x7f1384003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1397fff640 1 -- 172.21.15.111:0/380707362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1384003520 con 0x7f1384004d40 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1397fff640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13840051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1390001fa0 tx=0x7f1384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2261ef660dd10c0d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390001bf0 con 0x7f1384004d40 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1390003b70 con 0x7f1384004d40 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.399+0000 7f1396ffd640 1 -- 172.21.15.111:0/380707362 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13900050d0 con 0x7f1384004d40 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f139e1c4640 1 -- 172.21.15.111:0/380707362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 msgr2=0x7f13840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f139e1c4640 1 --2- 172.21.15.111:0/380707362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13840051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1390001fa0 tx=0x7f1384005830 comp rx=0 tx=0).stop 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f1397fff640 1 -- 172.21.15.111:0/380707362 reap_dead start 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f139e1c4640 1 -- 172.21.15.111:0/380707362 shutdown_connections 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f139e1c4640 1 -- 172.21.15.111:0/380707362 >> 172.21.15.111:0/380707362 conn(0x7f13980d42c0 msgr2=0x7f13980d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f139e1c4640 1 -- 172.21.15.111:0/380707362 shutdown_connections 2024-05-27T00:39:18.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.400+0000 7f139e1c4640 1 -- 172.21.15.111:0/380707362 wait complete. 2024-05-27T00:39:18.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.401+0000 7f139e1c4640 1 Processor -- start 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.401+0000 7f139e1c4640 1 -- start start 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f139e1c4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13980c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f139e1c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13980c8600 0x7f13981f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f139e1c4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980ca490 0x7f1398146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f139e1c4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f13980d34a0 con 0x7f1384004d40 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f139e1c4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f13980d3320 con 0x7f13980c8600 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f139e1c4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f13980d3620 con 0x7f13980ca490 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13980c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13980c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47912/0 (socket says 172.21.15.111:47912) 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 -- 172.21.15.111:0/2990154550 learned_addr learned my addr 172.21.15.111:0/2990154550 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 -- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980ca490 msgr2=0x7f1398146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980ca490 0x7f1398146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 -- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13980c8600 msgr2=0x7f13981f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f138ffff640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13980c8600 0x7f13981f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13980c8600 0x7f13981f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 -- 172.21.15.111:0/2990154550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13980c8b50 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.402+0000 7f13977fe640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13980c9bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1388011920 tx=0x7f1388011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.403+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138800faf0 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.403+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138800fc90 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.403+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13980c8e30 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.403+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388014440 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.403+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13980c93f0 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.404+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1398068680 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.404+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1388012040 con 0x7f1384004d40 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.404+0000 7f1394ff9640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f137806be30 0x7f137806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.405+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f138808b280 con 0x7f1384004d40 2024-05-27T00:39:18.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.405+0000 7f138ffff640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f137806be30 0x7f137806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.405+0000 7f138ffff640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f137806be30 0x7f137806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1398160c00 tx=0x7f1380009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.409+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1388019030 con 0x7f1384004d40 2024-05-27T00:39:18.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.584+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f13980cef10 con 0x7f1384004d40 2024-05-27T00:39:18.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.585+0000 7f1394ff9640 1 -- 172.21.15.111:0/2990154550 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f1388059ef0 con 0x7f1384004d40 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:39:18.586 INFO:teuthology.orchestra.run.smithi111.stdout:] 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.586+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f137806be30 msgr2=0x7f137806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.586+0000 7f139e1c4640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f137806be30 0x7f137806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1398160c00 tx=0x7f1380009040 comp rx=0 tx=0).stop 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 msgr2=0x7f13980c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13980c9bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1388011920 tx=0x7f1388011de0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 shutdown_connections 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f137806be30 0x7f137806e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13980ca490 0x7f1398146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13980c8600 0x7f13981f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 --2- 172.21.15.111:0/2990154550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1384004d40 0x7f13980c9bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 >> 172.21.15.111:0/2990154550 conn(0x7f13980d42c0 msgr2=0x7f13980d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:18.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 shutdown_connections 2024-05-27T00:39:18.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.587+0000 7f139e1c4640 1 -- 172.21.15.111:0/2990154550 wait complete. 2024-05-27T00:39:18.598 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T00:39:18.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T00:39:18.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.700+0000 7f9916658640 1 Processor -- start 2024-05-27T00:39:18.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.701+0000 7f9916658640 1 -- start start 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.701+0000 7f9916658640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99100d3440 0x7f99100b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.701+0000 7f9916658640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f991006c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.701+0000 7f9916658640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991006b840 con 0x7f9910161c10 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.701+0000 7f9916658640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f991006bb20 con 0x7f99100d3440 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.701+0000 7f9916658640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f991006be00 con 0x7f99100b34d0 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99100d3440 0x7f99100b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f9914bf5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f991006c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99100d3440 0x7f99100b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56346/0 (socket says 172.21.15.111:56346) 2024-05-27T00:39:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 -- 172.21.15.111:0/2908075896 learned_addr learned my addr 172.21.15.111:0/2908075896 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f9914bf5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f991006c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:32976/0 (socket says 172.21.15.111:32976) 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990ffff640 1 --1- 172.21.15.111:0/2908075896 >> v1:172.21.15.153:6789/0 conn(0x7f9910161c10 0x7f99100d2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:38822/0 (socket says 172.21.15.111:38822) 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 -- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 msgr2=0x7f991006c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 --2- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f991006c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 -- 172.21.15.111:0/2908075896 >> v1:172.21.15.153:6789/0 conn(0x7f9910161c10 legacy=0x7f99100d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990f7fe640 1 -- 172.21.15.111:0/2908075896 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991006c0e0 con 0x7f99100d3440 2024-05-27T00:39:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.702+0000 7f990effd640 1 -- 172.21.15.111:0/2908075896 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226731634 0 0) 0x7f991006b840 con 0x7f9910161c10 2024-05-27T00:39:18.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.703+0000 7f990f7fe640 1 --2- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99100d3440 0x7f99100b2f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9904009870 tx=0x7f990402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b7339a9e5eb70c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.703+0000 7f990effd640 1 -- 172.21.15.111:0/2908075896 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904037070 con 0x7f99100d3440 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.703+0000 7f990effd640 1 -- 172.21.15.111:0/2908075896 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032440 con 0x7f99100d3440 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.703+0000 7f990effd640 1 -- 172.21.15.111:0/2908075896 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904032810 con 0x7f99100d3440 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.703+0000 7f9916658640 1 -- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99100d3440 msgr2=0x7f99100b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.703+0000 7f9916658640 1 --2- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99100d3440 0x7f99100b2f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9904009870 tx=0x7f990402a270 comp rx=0 tx=0).stop 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.704+0000 7f9916658640 1 -- 172.21.15.111:0/2908075896 shutdown_connections 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.704+0000 7f9916658640 1 --2- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f991006c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.704+0000 7f9916658640 1 --2- 172.21.15.111:0/2908075896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99100d3440 0x7f99100b2f30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.704+0000 7f9916658640 1 -- 172.21.15.111:0/2908075896 >> 172.21.15.111:0/2908075896 conn(0x7f99100b5c40 msgr2=0x7f99100b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.704+0000 7f9916658640 1 -- 172.21.15.111:0/2908075896 shutdown_connections 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.704+0000 7f9916658640 1 -- 172.21.15.111:0/2908075896 wait complete. 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.705+0000 7f9916658640 1 Processor -- start 2024-05-27T00:39:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.705+0000 7f9916658640 1 -- start start 2024-05-27T00:39:18.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f99101532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 0x7f9910150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9910161c10 0x7f99100b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f99100d0830 con 0x7f99100d3440 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99100d06b0 con 0x7f9910161c10 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f99100b7340 con 0x7f99100b34d0 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 0x7f9910150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 0x7f9910150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47926/0 (socket says 172.21.15.111:47926) 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 -- 172.21.15.111:0/686866984 learned_addr learned my addr 172.21.15.111:0/686866984 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9914bf5640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9910161c10 0x7f99100b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990ffff640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f99101532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 -- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 msgr2=0x7f99101532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f99101532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 -- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9910161c10 msgr2=0x7f99100b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:18.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9910161c10 0x7f99100b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 -- 172.21.15.111:0/686866984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99101537f0 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990ffff640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f99101532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990f7fe640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 0x7f9910150480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f99040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904037040 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904042070 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101514a0 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.706+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904030880 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.707+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9910151a60 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.708+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9910068680 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.708+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9904030a20 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.708+0000 7f990cff9640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98fc06be30 0x7f98fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.709+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f99040ab2a0 con 0x7f99100d3440 2024-05-27T00:39:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.709+0000 7f990ffff640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98fc06be30 0x7f98fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.709+0000 7f990ffff640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98fc06be30 0x7f98fc06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9900000f60 tx=0x7f9900007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.713+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9904079fa0 con 0x7f99100d3440 2024-05-27T00:39:18.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:18.884+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f99100b5900 con 0x7f99100d3440 2024-05-27T00:39:19.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.551+0000 7f990cff9640 1 -- 172.21.15.111:0/686866984 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f99040798a0 con 0x7f99100d3440 2024-05-27T00:39:19.552 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.553+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98fc06be30 msgr2=0x7f98fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.553+0000 7f9916658640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98fc06be30 0x7f98fc06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9900000f60 tx=0x7f9900007cc0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.553+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 msgr2=0x7f9910150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.553+0000 7f9916658640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 0x7f9910150480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f99040046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 shutdown_connections 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98fc06be30 0x7f98fc06e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9910161c10 0x7f99100b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99100d3440 0x7f9910150480 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 --2- 172.21.15.111:0/686866984 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99100b34d0 0x7f99101532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 >> 172.21.15.111:0/686866984 conn(0x7f99100b5c40 msgr2=0x7f99100b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:19.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 shutdown_connections 2024-05-27T00:39:19.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.554+0000 7f9916658640 1 -- 172.21.15.111:0/686866984 wait complete. 2024-05-27T00:39:19.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-27T00:39:19.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 Processor -- start 2024-05-27T00:39:19.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 -- start start 2024-05-27T00:39:19.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c1459c0 0x7f8a5c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c0d19f0 0x7f8a5c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8a5c0ceeb0 0x7f8a5c15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5c0c7700 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5c0c79e0 con 0x7f8a5c0ceeb0 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.683+0000 7f8a61c53640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c0c7cc0 con 0x7f8a5c1459c0 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c0d19f0 0x7f8a5c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a5c0d19f0 0x7f8a5c0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47940/0 (socket says 172.21.15.111:47940) 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 -- 172.21.15.111:0/2395586085 learned_addr learned my addr 172.21.15.111:0/2395586085 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5bfff640 1 --2- 172.21.15.111:0/2395586085 >> v2:172.21.15.153:3300/0 conn(0x7f8a5c0ceeb0 0x7f8a5c15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2395586085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c1459c0 0x7f8a5c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 -- 172.21.15.111:0/2395586085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c1459c0 msgr2=0x7f8a5c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2395586085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c1459c0 0x7f8a5c0c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 -- 172.21.15.111:0/2395586085 >> v2:172.21.15.153:3300/0 conn(0x7f8a5c0ceeb0 msgr2=0x7f8a5c15fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2395586085 >> v2:172.21.15.153:3300/0 conn(0x7f8a5c0ceeb0 0x7f8a5c15fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5affd640 1 -- 172.21.15.111:0/2395586085 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c0c7fa0 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.684+0000 7f8a5bfff640 1 --2- 172.21.15.111:0/2395586085 >> v2:172.21.15.153:3300/0 conn(0x7f8a5c0ceeb0 0x7f8a5c15fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2395586085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c0ce970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0d2b60 tx=0x7f8a5002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b77bdb376e9a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a5a7fc640 1 -- 172.21.15.111:0/2395586085 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50037070 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a5a7fc640 1 -- 172.21.15.111:0/2395586085 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a50032440 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a5a7fc640 1 -- 172.21.15.111:0/2395586085 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50032810 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 -- 172.21.15.111:0/2395586085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 msgr2=0x7f8a5c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2395586085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c0ce970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0d2b60 tx=0x7f8a5002a140 comp rx=0 tx=0).stop 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 -- 172.21.15.111:0/2395586085 shutdown_connections 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2395586085 >> v2:172.21.15.153:3300/0 conn(0x7f8a5c0ceeb0 0x7f8a5c15fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2395586085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c0ce970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2395586085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c1459c0 0x7f8a5c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 -- 172.21.15.111:0/2395586085 >> 172.21.15.111:0/2395586085 conn(0x7f8a5c0d1f60 msgr2=0x7f8a5c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:19.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 -- 172.21.15.111:0/2395586085 shutdown_connections 2024-05-27T00:39:19.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.685+0000 7f8a61c53640 1 -- 172.21.15.111:0/2395586085 wait complete. 2024-05-27T00:39:19.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 Processor -- start 2024-05-27T00:39:19.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 -- start start 2024-05-27T00:39:19.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c0ceeb0 0x7f8a5c133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c1e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5c1459c0 0x7f8a5c06df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c145f20 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c145da0 con 0x7f8a5c1459c0 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.687+0000 7f8a61c53640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5c1601f0 con 0x7f8a5c0ceeb0 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c1e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c1e54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47952/0 (socket says 172.21.15.111:47952) 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 -- 172.21.15.111:0/2246933697 learned_addr learned my addr 172.21.15.111:0/2246933697 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5bfff640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5c1459c0 0x7f8a5c06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c0ceeb0 0x7f8a5c133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 -- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c0ceeb0 msgr2=0x7f8a5c133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c0ceeb0 0x7f8a5c133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 -- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5c1459c0 msgr2=0x7f8a5c06df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5c1459c0 0x7f8a5c06df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 -- 172.21.15.111:0/2246933697 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c06ce60 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a5affd640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c1e54f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a180 tx=0x7f8a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50037040 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a50042070 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50030bb0 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c06d140 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.688+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c06d700 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.689+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c068680 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.690+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a50047460 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.690+0000 7f8a43fff640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8a4806be60 0x7f8a4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.690+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f8a500ab6c0 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.691+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8a4806be60 0x7f8a4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:19.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.691+0000 7f8a5b7fe640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8a4806be60 0x7f8a4806e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00a9a0 tx=0x7f8a4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:19.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.695+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8a50079c90 con 0x7f8a5c0d19f0 2024-05-27T00:39:19.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:19.868+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f8a5c06fcc0 con 0x7f8a5c0d19f0 2024-05-27T00:39:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.553+0000 7f8a43fff640 1 -- 172.21.15.111:0/2246933697 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f8a5007df90 con 0x7f8a5c0d19f0 2024-05-27T00:39:20.554 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8a4806be60 msgr2=0x7f8a4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8a4806be60 0x7f8a4806e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00a9a0 tx=0x7f8a4c008040 comp rx=0 tx=0).stop 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 msgr2=0x7f8a5c1e54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c1e54f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a180 tx=0x7f8a500046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 shutdown_connections 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8a4806be60 0x7f8a4806e320 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00a9a0 tx=0x7f8a4c008040 comp rx=0 tx=0).stop 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a5c1459c0 0x7f8a5c06df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a5c0d19f0 0x7f8a5c1e54f0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a180 tx=0x7f8a500046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 --2- 172.21.15.111:0/2246933697 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a5c0ceeb0 0x7f8a5c133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.557+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 >> 172.21.15.111:0/2246933697 conn(0x7f8a5c0d1f60 msgr2=0x7f8a5c15f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:20.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.561+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 shutdown_connections 2024-05-27T00:39:20.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.562+0000 7f8a61c53640 1 -- 172.21.15.111:0/2246933697 wait complete. 2024-05-27T00:39:20.583 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T00:39:20.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.686+0000 7f057de12640 1 Processor -- start 2024-05-27T00:39:20.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.686+0000 7f057de12640 1 -- start start 2024-05-27T00:39:20.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f057de12640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f057815e830 0x7f05781459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:20.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f057de12640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0578070030 0x7f05780704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:20.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f057de12640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0578145f90 con 0x7f0578144a10 2024-05-27T00:39:20.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f057de12640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f05780709e0 con 0x7f057815e830 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f057de12640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0578146270 con 0x7f0578070030 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f057815e830 0x7f05781459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f057815e830 0x7f05781459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58790/0 (socket says 172.21.15.111:58790) 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 -- 172.21.15.111:0/2008344030 learned_addr learned my addr 172.21.15.111:0/2008344030 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0577fff640 1 --2- 172.21.15.111:0/2008344030 >> v2:172.21.15.153:3300/0 conn(0x7f0578070030 0x7f05780704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 -- 172.21.15.111:0/2008344030 >> v2:172.21.15.153:3300/0 conn(0x7f0578070030 msgr2=0x7f05780704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 --2- 172.21.15.111:0/2008344030 >> v2:172.21.15.153:3300/0 conn(0x7f0578070030 0x7f05780704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 -- 172.21.15.111:0/2008344030 >> v1:172.21.15.111:6789/0 conn(0x7f0578144a10 legacy=0x7f05780a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:20.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f05767fc640 1 -- 172.21.15.111:0/2008344030 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8145562 0 0) 0x7f0578145f90 con 0x7f0578144a10 2024-05-27T00:39:20.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.687+0000 7f0576ffd640 1 -- 172.21.15.111:0/2008344030 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0578146550 con 0x7f057815e830 2024-05-27T00:39:20.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.688+0000 7f0576ffd640 1 --2- 172.21.15.111:0/2008344030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f057815e830 0x7f05781459b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f056c009870 tx=0x7f056c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d666633cde64530b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:20.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.688+0000 7f05767fc640 1 -- 172.21.15.111:0/2008344030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c037070 con 0x7f057815e830 2024-05-27T00:39:20.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.688+0000 7f05767fc640 1 -- 172.21.15.111:0/2008344030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c032440 con 0x7f057815e830 2024-05-27T00:39:20.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.688+0000 7f05767fc640 1 -- 172.21.15.111:0/2008344030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c032810 con 0x7f057815e830 2024-05-27T00:39:20.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 -- 172.21.15.111:0/2008344030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f057815e830 msgr2=0x7f05781459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:20.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 --2- 172.21.15.111:0/2008344030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f057815e830 0x7f05781459b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f056c009870 tx=0x7f056c02a2a0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 -- 172.21.15.111:0/2008344030 shutdown_connections 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 --2- 172.21.15.111:0/2008344030 >> v2:172.21.15.153:3300/0 conn(0x7f0578070030 0x7f05780704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 --2- 172.21.15.111:0/2008344030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f057815e830 0x7f05781459b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 -- 172.21.15.111:0/2008344030 >> 172.21.15.111:0/2008344030 conn(0x7f0578073130 msgr2=0x7f05780b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 -- 172.21.15.111:0/2008344030 shutdown_connections 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.689+0000 7f057de12640 1 -- 172.21.15.111:0/2008344030 wait complete. 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.690+0000 7f057de12640 1 Processor -- start 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.690+0000 7f057de12640 1 -- start start 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f057de12640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 0x7f057813a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f057de12640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0578144a10 0x7f0578138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:20.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f057de12640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057815e830 0x7f057815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f057de12640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0578147030 con 0x7f0578070030 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f057de12640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0578146b10 con 0x7f057815e830 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f057de12640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f05781471b0 con 0x7f0578144a10 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 0x7f057813a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 0x7f057813a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58798/0 (socket says 172.21.15.111:58798) 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 -- 172.21.15.111:0/3954414123 learned_addr learned my addr 172.21.15.111:0/3954414123 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f0576ffd640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0578144a10 0x7f0578138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 -- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0578144a10 msgr2=0x7f0578138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0578144a10 0x7f0578138a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 -- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057815e830 msgr2=0x7f057815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057815e830 0x7f057815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 -- 172.21.15.111:0/3954414123 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0578139270 con 0x7f0578070030 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f05777fe640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 0x7f057813a9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f056800c8c0 tx=0x7f056800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568017070 con 0x7f0578070030 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.691+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568003dc0 con 0x7f0578070030 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.692+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0578139550 con 0x7f0578070030 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.692+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05680052b0 con 0x7f0578070030 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.692+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0578138f60 con 0x7f0578070030 2024-05-27T00:39:20.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.693+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0578068680 con 0x7f0578070030 2024-05-27T00:39:20.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.693+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f056801a070 con 0x7f0578070030 2024-05-27T00:39:20.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.694+0000 7f055ffff640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f056406be30 0x7f056406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:20.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.694+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f056808a4f0 con 0x7f0578070030 2024-05-27T00:39:20.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.694+0000 7f0576ffd640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f056406be30 0x7f056406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:20.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.694+0000 7f0576ffd640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f056406be30 0x7f056406e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f056c034000 tx=0x7f056c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:20.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.698+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0568058f90 con 0x7f0578070030 2024-05-27T00:39:20.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:20.906+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0578141da0 con 0x7f0578070030 2024-05-27T00:39:21.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.576+0000 7f055ffff640 1 -- 172.21.15.111:0/3954414123 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f0568058890 con 0x7f0578070030 2024-05-27T00:39:21.579 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T00:39:21.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f056406be30 msgr2=0x7f056406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f056406be30 0x7f056406e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f056c034000 tx=0x7f056c02e000 comp rx=0 tx=0).stop 2024-05-27T00:39:21.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 msgr2=0x7f057813a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 0x7f057813a9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f056800c8c0 tx=0x7f056800cd80 comp rx=0 tx=0).stop 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 shutdown_connections 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f056406be30 0x7f056406e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057815e830 0x7f057815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0578144a10 0x7f0578138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.582+0000 7f057de12640 1 --2- 172.21.15.111:0/3954414123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0578070030 0x7f057813a9f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.583+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 >> 172.21.15.111:0/3954414123 conn(0x7f0578073130 msgr2=0x7f057815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:21.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.584+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 shutdown_connections 2024-05-27T00:39:21.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.584+0000 7f057de12640 1 -- 172.21.15.111:0/3954414123 wait complete. 2024-05-27T00:39:21.605 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-27T00:39:21.606 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-27T00:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.711+0000 7fb41f15a640 1 Processor -- start 2024-05-27T00:39:21.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.711+0000 7fb41f15a640 1 -- start start 2024-05-27T00:39:21.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.711+0000 7fb41f15a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb4180d3440 0x7fb4180b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.711+0000 7fb41f15a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41806b840 con 0x7fb418161c10 2024-05-27T00:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.711+0000 7fb41f15a640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41806bb20 con 0x7fb4180b34d0 2024-05-27T00:39:21.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.711+0000 7fb41f15a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb41806be00 con 0x7fb4180d3440 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb417fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb4180d3440 0x7fb4180b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb417fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb4180d3440 0x7fb4180b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58814/0 (socket says 172.21.15.111:58814) 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb41cef6640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb418161c10 0x7fb4180d2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35346/0 (socket says 172.21.15.111:35346) 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb417fff640 1 -- 172.21.15.111:0/2946386291 learned_addr learned my addr 172.21.15.111:0/2946386291 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb417fff640 1 -- 172.21.15.111:0/2946386291 >> v1:172.21.15.153:6789/0 conn(0x7fb4180b34d0 legacy=0x7fb4180b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb417fff640 1 -- 172.21.15.111:0/2946386291 >> v1:172.21.15.111:6789/0 conn(0x7fb418161c10 legacy=0x7fb4180d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb4177fe640 1 -- 172.21.15.111:0/2946386291 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2190778737 0 0) 0x7fb41806b840 con 0x7fb418161c10 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.712+0000 7fb417fff640 1 -- 172.21.15.111:0/2946386291 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41806c0e0 con 0x7fb4180d3440 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb417fff640 1 --2- 172.21.15.111:0/2946386291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4180d3440 0x7fb4180b2f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e50 tx=0x7fb40c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e8ab6f84fe541a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb4177fe640 1 -- 172.21.15.111:0/2946386291 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c036070 con 0x7fb4180d3440 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb4177fe640 1 -- 172.21.15.111:0/2946386291 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c031600 con 0x7fb4180d3440 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb4177fe640 1 -- 172.21.15.111:0/2946386291 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0319f0 con 0x7fb4180d3440 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 -- 172.21.15.111:0/2946386291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4180d3440 msgr2=0x7fb4180b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 --2- 172.21.15.111:0/2946386291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4180d3440 0x7fb4180b2f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e50 tx=0x7fb40c02a7c0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 -- 172.21.15.111:0/2946386291 shutdown_connections 2024-05-27T00:39:21.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 --2- 172.21.15.111:0/2946386291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4180d3440 0x7fb4180b2f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 -- 172.21.15.111:0/2946386291 >> 172.21.15.111:0/2946386291 conn(0x7fb4180b5c40 msgr2=0x7fb4180b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 -- 172.21.15.111:0/2946386291 shutdown_connections 2024-05-27T00:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.713+0000 7fb41f15a640 1 -- 172.21.15.111:0/2946386291 wait complete. 2024-05-27T00:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.714+0000 7fb41f15a640 1 Processor -- start 2024-05-27T00:39:21.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.714+0000 7fb41f15a640 1 -- start start 2024-05-27T00:39:21.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41f15a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180b34d0 0x7fb418159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:21.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41f15a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180d3440 0x7fb418158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41f15a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 0x7fb4180b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41f15a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4180d0830 con 0x7fb418161c10 2024-05-27T00:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41f15a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb4180d06b0 con 0x7fb4180b34d0 2024-05-27T00:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41f15a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb41806c8e0 con 0x7fb4180d3440 2024-05-27T00:39:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41d6f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 0x7fb4180b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41d6f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 0x7fb4180b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58830/0 (socket says 172.21.15.111:58830) 2024-05-27T00:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.715+0000 7fb41d6f7640 1 -- 172.21.15.111:0/3505549686 learned_addr learned my addr 172.21.15.111:0/3505549686 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41cef6640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180b34d0 0x7fb418159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:21.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb417fff640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180d3440 0x7fb418158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41d6f7640 1 -- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180d3440 msgr2=0x7fb418158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41d6f7640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180d3440 0x7fb418158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41d6f7640 1 -- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180b34d0 msgr2=0x7fb418159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41d6f7640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180b34d0 0x7fb418159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41d6f7640 1 -- 172.21.15.111:0/3505549686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb418149840 con 0x7fb418161c10 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb417fff640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180d3440 0x7fb418158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:21.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41cef6640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180b34d0 0x7fb418159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41d6f7640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 0x7fb4180b7c30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c390 tx=0x7fb41000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410017070 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4100052d0 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb418149b20 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100037f0 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.716+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41814a0e0 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.717+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41806ccb0 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.718+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb41001a070 con 0x7fb418161c10 2024-05-27T00:39:21.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.718+0000 7fb4157fa640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40406be30 0x7fb40406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.718+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fb410089fd0 con 0x7fb418161c10 2024-05-27T00:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.718+0000 7fb41cef6640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40406be30 0x7fb40406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.719+0000 7fb41cef6640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40406be30 0x7fb40406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb408007fd0 tx=0x7fb408007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:21.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.723+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb41806ccb0 con 0x7fb418161c10 2024-05-27T00:39:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:21.905+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb418068680 con 0x7fb418161c10 2024-05-27T00:39:22.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.581+0000 7fb4157fa640 1 -- 172.21.15.111:0/3505549686 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7fb410058330 con 0x7fb418161c10 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40406be30 msgr2=0x7fb40406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40406be30 0x7fb40406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb408007fd0 tx=0x7fb408007de0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 msgr2=0x7fb4180b7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 0x7fb4180b7c30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb41000c390 tx=0x7fb41000c850 comp rx=0 tx=0).stop 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 shutdown_connections 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40406be30 0x7fb40406e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb418161c10 0x7fb4180b7c30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb4180d3440 0x7fb418158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 --2- 172.21.15.111:0/3505549686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4180b34d0 0x7fb418159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 >> 172.21.15.111:0/3505549686 conn(0x7fb4180b5c40 msgr2=0x7fb4180b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:22.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 shutdown_connections 2024-05-27T00:39:22.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.585+0000 7fb41f15a640 1 -- 172.21.15.111:0/3505549686 wait complete. 2024-05-27T00:39:22.600 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T00:39:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.712+0000 7f33a2253640 1 Processor -- start 2024-05-27T00:39:22.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.712+0000 7f33a2253640 1 -- start start 2024-05-27T00:39:22.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f33a2253640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c147010 0x7f339c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f33a2253640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 0x7f339c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f33a2253640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f339c0b2b10 con 0x7f339c0ced10 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f33a2253640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f339c1473f0 con 0x7f339c0b25a0 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f33a2253640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f339c0cf140 con 0x7f339c147010 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f339affd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 0x7f339c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f339b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c147010 0x7f339c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f339affd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 0x7f339c0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:58540/0 (socket says 172.21.15.111:58540) 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f339affd640 1 -- 172.21.15.111:0/1174768919 learned_addr learned my addr 172.21.15.111:0/1174768919 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f339b7fe640 1 -- 172.21.15.111:0/1174768919 >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 msgr2=0x7f339c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.713+0000 7f339b7fe640 1 --2- 172.21.15.111:0/1174768919 >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 0x7f339c0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339b7fe640 1 -- 172.21.15.111:0/1174768919 >> v1:172.21.15.153:6789/0 conn(0x7f339c0ced10 legacy=0x7f339c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339b7fe640 1 -- 172.21.15.111:0/1174768919 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c0b2df0 con 0x7f339c147010 2024-05-27T00:39:22.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339affd640 1 --2- 172.21.15.111:0/1174768919 >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 0x7f339c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339b7fe640 1 --2- 172.21.15.111:0/1174768919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c147010 0x7f339c06df10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebdee6823e66eba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339a7fc640 1 -- 172.21.15.111:0/1174768919 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c007740 con 0x7f339c147010 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339a7fc640 1 -- 172.21.15.111:0/1174768919 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338c0078e0 con 0x7f339c147010 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.714+0000 7f339a7fc640 1 -- 172.21.15.111:0/1174768919 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c007cb0 con 0x7f339c147010 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 -- 172.21.15.111:0/1174768919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c147010 msgr2=0x7f339c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 --2- 172.21.15.111:0/1174768919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c147010 0x7f339c06df10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c02a660 comp rx=0 tx=0).stop 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 -- 172.21.15.111:0/1174768919 shutdown_connections 2024-05-27T00:39:22.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 --2- 172.21.15.111:0/1174768919 >> v2:172.21.15.153:3300/0 conn(0x7f339c0b25a0 0x7f339c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 --2- 172.21.15.111:0/1174768919 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c147010 0x7f339c06df10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 -- 172.21.15.111:0/1174768919 >> 172.21.15.111:0/1174768919 conn(0x7f339c0d1f50 msgr2=0x7f339c0d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 -- 172.21.15.111:0/1174768919 shutdown_connections 2024-05-27T00:39:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.715+0000 7f33a2253640 1 -- 172.21.15.111:0/1174768919 wait complete. 2024-05-27T00:39:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.716+0000 7f33a2253640 1 Processor -- start 2024-05-27T00:39:22.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.716+0000 7f33a2253640 1 -- start start 2024-05-27T00:39:22.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f33a2253640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c0b25a0 0x7f339c140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:22.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f33a2253640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 0x7f339c15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:22.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f33a2253640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c147010 0x7f339c0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:22.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f33a2253640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f339c0b3690 con 0x7f339c0ced10 2024-05-27T00:39:22.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f33a2253640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f339c0b33b0 con 0x7f339c147010 2024-05-27T00:39:22.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f33a2253640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f339c0d1410 con 0x7f339c0b25a0 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f339affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 0x7f339c15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f339affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 0x7f339c15fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58842/0 (socket says 172.21.15.111:58842) 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f339affd640 1 -- 172.21.15.111:0/2077113588 learned_addr learned my addr 172.21.15.111:0/2077113588 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f339bfff640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c147010 0x7f339c0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.717+0000 7f339b7fe640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c0b25a0 0x7f339c140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f339affd640 1 -- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c0b25a0 msgr2=0x7f339c140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f339affd640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c0b25a0 0x7f339c140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f339affd640 1 -- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c147010 msgr2=0x7f339c0b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f339affd640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c147010 0x7f339c0b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f339affd640 1 -- 172.21.15.111:0/2077113588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c15ec70 con 0x7f339c0ced10 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f339affd640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 0x7f339c15fd10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f339000ea00 tx=0x7f339000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390017070 con 0x7f339c0ced10 2024-05-27T00:39:22.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339000ca60 con 0x7f339c0ced10 2024-05-27T00:39:22.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c15ef50 con 0x7f339c0ced10 2024-05-27T00:39:22.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339000ce10 con 0x7f339c0ced10 2024-05-27T00:39:22.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.718+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c15f510 con 0x7f339c0ced10 2024-05-27T00:39:22.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.719+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c068610 con 0x7f339c0ced10 2024-05-27T00:39:22.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.719+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f339001a070 con 0x7f339c0ced10 2024-05-27T00:39:22.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.720+0000 7f3383fff640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f338806be60 0x7f338806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:22.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.720+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f339008b290 con 0x7f339c0ced10 2024-05-27T00:39:22.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.720+0000 7f339b7fe640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f338806be60 0x7f338806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:22.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.720+0000 7f339b7fe640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f338806be60 0x7f338806e320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:22.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.724+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3390059a90 con 0x7f339c0ced10 2024-05-27T00:39:22.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:22.897+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f339c06dba0 con 0x7f339c0ced10 2024-05-27T00:39:24.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.594+0000 7f3383fff640 1 -- 172.21.15.111:0/2077113588 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f3390059390 con 0x7f339c0ced10 2024-05-27T00:39:24.595 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data_ec' created 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f338806be60 msgr2=0x7f338806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f338806be60 0x7f338806e320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c033040 comp rx=0 tx=0).stop 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 msgr2=0x7f339c15fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 0x7f339c15fd10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f339000ea00 tx=0x7f339000eec0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 shutdown_connections 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f338806be60 0x7f338806e320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f339c147010 0x7f339c0b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f339c0ced10 0x7f339c15fd10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 --2- 172.21.15.111:0/2077113588 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f339c0b25a0 0x7f339c140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 >> 172.21.15.111:0/2077113588 conn(0x7f339c0d1f50 msgr2=0x7f339c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:24.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 shutdown_connections 2024-05-27T00:39:24.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.598+0000 7f33a2253640 1 -- 172.21.15.111:0/2077113588 wait complete. 2024-05-27T00:39:24.618 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-27T00:39:24.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.726+0000 7fceef815640 1 Processor -- start 2024-05-27T00:39:24.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.726+0000 7fceef815640 1 -- start start 2024-05-27T00:39:24.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceef815640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcee8140ef0 0x7fcee815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceef815640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcee81412d0 0x7fcee81626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceef815640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8141a00 0x7fcee8144050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceef815640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fcee8141780 con 0x7fcee8140ef0 2024-05-27T00:39:24.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceef815640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcee815eea0 con 0x7fcee81412d0 2024-05-27T00:39:24.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceef815640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcee815f180 con 0x7fcee8141a00 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcee8140ef0 0x7fcee815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcee8140ef0 0x7fcee815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58844/0 (socket says 172.21.15.111:58844) 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 -- 172.21.15.111:0/748689989 learned_addr learned my addr 172.21.15.111:0/748689989 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceeddb2640 1 --2- 172.21.15.111:0/748689989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8141a00 0x7fcee8144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceecdb0640 1 --2- 172.21.15.111:0/748689989 >> v2:172.21.15.153:3300/0 conn(0x7fcee81412d0 0x7fcee81626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 -- 172.21.15.111:0/748689989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8141a00 msgr2=0x7fcee8144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 --2- 172.21.15.111:0/748689989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8141a00 0x7fcee8144050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 -- 172.21.15.111:0/748689989 >> v2:172.21.15.153:3300/0 conn(0x7fcee81412d0 msgr2=0x7fcee81626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 --2- 172.21.15.111:0/748689989 >> v2:172.21.15.153:3300/0 conn(0x7fcee81412d0 0x7fcee81626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.727+0000 7fceed5b1640 1 -- 172.21.15.111:0/748689989 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee815f460 con 0x7fcee8140ef0 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fceed5b1640 1 --2- 172.21.15.111:0/748689989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee8140ef0 0x7fcee815e850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcee8070790 tx=0x7fcee402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3624270628c4aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fcedffff640 1 -- 172.21.15.111:0/748689989 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4037070 con 0x7fcee8140ef0 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fcedffff640 1 -- 172.21.15.111:0/748689989 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee4032440 con 0x7fcee8140ef0 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fcedffff640 1 -- 172.21.15.111:0/748689989 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4032810 con 0x7fcee8140ef0 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fceef815640 1 -- 172.21.15.111:0/748689989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee8140ef0 msgr2=0x7fcee815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fceef815640 1 --2- 172.21.15.111:0/748689989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee8140ef0 0x7fcee815e850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcee8070790 tx=0x7fcee402a1e0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.728+0000 7fceef815640 1 -- 172.21.15.111:0/748689989 shutdown_connections 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.729+0000 7fceef815640 1 --2- 172.21.15.111:0/748689989 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8141a00 0x7fcee8144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.729+0000 7fceef815640 1 --2- 172.21.15.111:0/748689989 >> v2:172.21.15.153:3300/0 conn(0x7fcee81412d0 0x7fcee81626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.729+0000 7fceef815640 1 --2- 172.21.15.111:0/748689989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee8140ef0 0x7fcee815e850 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.729+0000 7fceef815640 1 -- 172.21.15.111:0/748689989 >> 172.21.15.111:0/748689989 conn(0x7fcee80ce840 msgr2=0x7fcee8072350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:24.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.729+0000 7fceef815640 1 -- 172.21.15.111:0/748689989 shutdown_connections 2024-05-27T00:39:24.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.729+0000 7fceef815640 1 -- 172.21.15.111:0/748689989 wait complete. 2024-05-27T00:39:24.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 Processor -- start 2024-05-27T00:39:24.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 -- start start 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8140ef0 0x7fcee81565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 0x7fcee815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee8141a00 0x7fcee815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcee8162c00 con 0x7fcee81412d0 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcee8143b30 con 0x7fcee8141a00 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.730+0000 7fceef815640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcee8141e50 con 0x7fcee8140ef0 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 0x7fcee815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 0x7fcee815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58858/0 (socket says 172.21.15.111:58858) 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 -- 172.21.15.111:0/3402718494 learned_addr learned my addr 172.21.15.111:0/3402718494 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceeddb2640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee8141a00 0x7fcee815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceed5b1640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8140ef0 0x7fcee81565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 -- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8140ef0 msgr2=0x7fcee81565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8140ef0 0x7fcee81565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 -- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee8141a00 msgr2=0x7fcee815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee8141a00 0x7fcee815bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 -- 172.21.15.111:0/3402718494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee81f43e0 con 0x7fcee81412d0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceecdb0640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 0x7fcee815b780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fced800d800 tx=0x7fced800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced800b6c0 con 0x7fcee81412d0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8019070 con 0x7fcee81412d0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee8156b30 con 0x7fcee81412d0 2024-05-27T00:39:24.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced800ba80 con 0x7fcee81412d0 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.731+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee81f4930 con 0x7fcee81412d0 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.732+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee8068680 con 0x7fcee81412d0 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.733+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fced800bc20 con 0x7fcee81412d0 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.733+0000 7fceddffb640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcecc06be30 0x7fcecc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.733+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fced808b4f0 con 0x7fcee81412d0 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.734+0000 7fceed5b1640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcecc06be30 0x7fcecc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:24.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.734+0000 7fceed5b1640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcecc06be30 0x7fcecc06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcee4034000 tx=0x7fcee402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:24.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.738+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcee8156b30 con 0x7fcee81412d0 2024-05-27T00:39:24.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:24.911+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fcee81615e0 con 0x7fcee81412d0 2024-05-27T00:39:25.476 INFO:tasks.ceph.mds.d.smithi153.stderr:2024-05-27T00:39:25.475+0000 7f7a2c5b4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T00:39:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.601+0000 7fceddffb640 1 -- 172.21.15.111:0/3402718494 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fced8059530 con 0x7fcee81412d0 2024-05-27T00:39:25.603 INFO:teuthology.orchestra.run.smithi111.stderr:set pool 4 allow_ec_overwrites to true 2024-05-27T00:39:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcecc06be30 msgr2=0x7fcecc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcecc06be30 0x7fcecc06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcee4034000 tx=0x7fcee402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 msgr2=0x7fcee815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:25.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 0x7fcee815b780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fced800d800 tx=0x7fced800dcc0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 shutdown_connections 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcecc06be30 0x7fcecc06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee8141a00 0x7fcee815bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcee81412d0 0x7fcee815b780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 --2- 172.21.15.111:0/3402718494 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcee8140ef0 0x7fcee81565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 >> 172.21.15.111:0/3402718494 conn(0x7fcee80ce840 msgr2=0x7fcee8143150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 shutdown_connections 2024-05-27T00:39:25.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.605+0000 7fceef815640 1 -- 172.21.15.111:0/3402718494 wait complete. 2024-05-27T00:39:25.631 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-27T00:39:25.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.733+0000 7f33207c8640 1 Processor -- start 2024-05-27T00:39:25.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.733+0000 7f33207c8640 1 -- start start 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f33207c8640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3318144000 0x7f33180b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f33207c8640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f33207c8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33180cdc80 0x7f33180726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f33207c8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f33180b42f0 con 0x7f33180cdc80 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f33207c8640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f33180b3440 con 0x7f3318144000 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f33207c8640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f33180b36e0 con 0x7f331806b390 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33180cdc80 0x7f33180726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33180cdc80 0x7f33180726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58870/0 (socket says 172.21.15.111:58870) 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 -- 172.21.15.111:0/1610703899 learned_addr learned my addr 172.21.15.111:0/1610703899 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331e564640 1 --2- 172.21.15.111:0/1610703899 >> v2:172.21.15.153:3300/0 conn(0x7f3318144000 0x7f33180b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331dd63640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 -- 172.21.15.111:0/1610703899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 msgr2=0x7f33180cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 -- 172.21.15.111:0/1610703899 >> v2:172.21.15.153:3300/0 conn(0x7f3318144000 msgr2=0x7f33180b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 --2- 172.21.15.111:0/1610703899 >> v2:172.21.15.153:3300/0 conn(0x7f3318144000 0x7f33180b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331ed65640 1 -- 172.21.15.111:0/1610703899 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33180b39c0 con 0x7f33180cdc80 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331e564640 1 --2- 172.21.15.111:0/1610703899 >> v2:172.21.15.153:3300/0 conn(0x7f3318144000 0x7f33180b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.734+0000 7f331dd63640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f331ed65640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33180cdc80 0x7f33180726d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f33180726a0 tx=0x7f331402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf1bd16e7a41e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f331d562640 1 -- 172.21.15.111:0/1610703899 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314037070 con 0x7f33180cdc80 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f331d562640 1 -- 172.21.15.111:0/1610703899 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314032440 con 0x7f33180cdc80 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f331d562640 1 -- 172.21.15.111:0/1610703899 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314032810 con 0x7f33180cdc80 2024-05-27T00:39:25.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f33207c8640 1 -- 172.21.15.111:0/1610703899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33180cdc80 msgr2=0x7f33180726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f33207c8640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33180cdc80 0x7f33180726d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f33180726a0 tx=0x7f331402a1e0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f33207c8640 1 -- 172.21.15.111:0/1610703899 shutdown_connections 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f33207c8640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33180cdc80 0x7f33180726d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f33207c8640 1 --2- 172.21.15.111:0/1610703899 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.735+0000 7f33207c8640 1 --2- 172.21.15.111:0/1610703899 >> v2:172.21.15.153:3300/0 conn(0x7f3318144000 0x7f33180b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.736+0000 7f33207c8640 1 -- 172.21.15.111:0/1610703899 >> 172.21.15.111:0/1610703899 conn(0x7f33180b1c00 msgr2=0x7f33180b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.736+0000 7f33207c8640 1 -- 172.21.15.111:0/1610703899 shutdown_connections 2024-05-27T00:39:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.736+0000 7f33207c8640 1 -- 172.21.15.111:0/1610703899 wait complete. 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 Processor -- start 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 -- start start 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33180cdc80 0x7f331813f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 0x7f33180c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3318072c10 con 0x7f3318144000 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33180b3f80 con 0x7f33180cdc80 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.737+0000 7f33207c8640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3318072d90 con 0x7f331806b390 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 0x7f33180c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 0x7f33180c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58882/0 (socket says 172.21.15.111:58882) 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 -- 172.21.15.111:0/18550046 learned_addr learned my addr 172.21.15.111:0/18550046 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331e564640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331dd63640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33180cdc80 0x7f331813f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 -- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 msgr2=0x7f33180c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 -- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33180cdc80 msgr2=0x7f331813f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33180cdc80 0x7f331813f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 -- 172.21.15.111:0/18550046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3318159c90 con 0x7f3318144000 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f331ed65640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 0x7f33180c6fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33140090b0 tx=0x7f33140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314037040 con 0x7f3318144000 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314042070 con 0x7f3318144000 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3318159f70 con 0x7f3318144000 2024-05-27T00:39:25.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314030920 con 0x7f3318144000 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.738+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331815a530 con 0x7f3318144000 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.739+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3318068680 con 0x7f3318144000 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.740+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3314030ac0 con 0x7f3318144000 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.740+0000 7f330affd640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f32fc06be60 0x7f32fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.740+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f33140ab900 con 0x7f3318144000 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.740+0000 7f331e564640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f32fc06be60 0x7f32fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:25.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.741+0000 7f331e564640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f32fc06be60 0x7f32fc06e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3310006fd0 tx=0x7f3310008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:25.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.745+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3314079940 con 0x7f3318144000 2024-05-27T00:39:25.958 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:25.956+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f33180ade10 con 0x7f3318144000 2024-05-27T00:39:26.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.602+0000 7f330affd640 1 -- 172.21.15.111:0/18550046 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f331407dc40 con 0x7f3318144000 2024-05-27T00:39:26.605 INFO:teuthology.orchestra.run.smithi111.stderr:added data pool 4 to fsmap 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f32fc06be60 msgr2=0x7f32fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f32fc06be60 0x7f32fc06e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3310006fd0 tx=0x7f3310008040 comp rx=0 tx=0).stop 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 msgr2=0x7f33180c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 0x7f33180c6fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33140090b0 tx=0x7f33140046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 shutdown_connections 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f32fc06be60 0x7f32fc06e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3318144000 0x7f33180c6fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33180cdc80 0x7f331813f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 --2- 172.21.15.111:0/18550046 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f331806b390 0x7f33180c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 >> 172.21.15.111:0/18550046 conn(0x7f33180b1c00 msgr2=0x7f33180cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:26.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 shutdown_connections 2024-05-27T00:39:26.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.605+0000 7f33207c8640 1 -- 172.21.15.111:0/18550046 wait complete. 2024-05-27T00:39:26.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:26.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.723+0000 7f6f50f90640 1 Processor -- start 2024-05-27T00:39:26.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.723+0000 7f6f50f90640 1 -- start start 2024-05-27T00:39:26.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f50f90640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f4c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f50f90640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c145f90 con 0x7f6f4c144a10 2024-05-27T00:39:26.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f50f90640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c146270 con 0x7f6f4c070030 2024-05-27T00:39:26.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f50f90640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c146550 con 0x7f6f4c15e830 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f4a59c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6f4c144a10 0x7f6f4c0a9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35362/0 (socket says 172.21.15.111:35362) 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f4a59c640 1 -- 172.21.15.111:0/350703576 learned_addr learned my addr 172.21.15.111:0/350703576 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f41d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f4c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865256532 0 0) 0x7f6f4c145f90 con 0x7f6f4c144a10 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f38003520 con 0x7f6f4c144a10 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 731213048 0 0) 0x7f6f38003520 con 0x7f6f4c144a10 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 msgr2=0x7f6f4c1459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f4c1459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 >> v1:172.21.15.153:6789/0 conn(0x7f6f4c070030 legacy=0x7f6f4c147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c146830 con 0x7f6f4c144a10 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f6f3c001670 con 0x7f6f4c144a10 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 >> v1:172.21.15.111:6789/0 conn(0x7f6f4c144a10 legacy=0x7f6f4c0a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f380039e0 0x7f6f38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f38004d40 0x7f6f380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c146550 con 0x7f6f380039e0 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3800baa0 con 0x7f6f38004d40 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3800bca0 con 0x7f6f4c15e830 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.724+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f380039e0 0x7f6f38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f41d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4ad9d640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f38004d40 0x7f6f380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4a59c640 1 -- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 msgr2=0x7f6f38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4a59c640 1 -- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f38004d40 msgr2=0x7f6f380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f38004d40 0x7f6f380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4a59c640 1 -- 172.21.15.111:0/350703576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f38003520 con 0x7f6f380039e0 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f41d9b640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c15e830 0x7f6f38004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4ad9d640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f38004d40 0x7f6f380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f380039e0 0x7f6f38003e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c001eb0 tx=0x7f6f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9571c1fc6928ad53 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.725+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c007d30 con 0x7f6f380039e0 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3c0058d0 con 0x7f6f380039e0 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f49d9b640 1 -- 172.21.15.111:0/350703576 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c005ce0 con 0x7f6f380039e0 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f50f90640 1 -- 172.21.15.111:0/350703576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f380039e0 msgr2=0x7f6f38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f50f90640 1 --2- 172.21.15.111:0/350703576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f380039e0 0x7f6f38003e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c001eb0 tx=0x7f6f38005830 comp rx=0 tx=0).stop 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f4ad9d640 1 -- 172.21.15.111:0/350703576 reap_dead start 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f50f90640 1 -- 172.21.15.111:0/350703576 shutdown_connections 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f50f90640 1 -- 172.21.15.111:0/350703576 >> 172.21.15.111:0/350703576 conn(0x7f6f4c073130 msgr2=0x7f6f4c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f50f90640 1 -- 172.21.15.111:0/350703576 shutdown_connections 2024-05-27T00:39:26.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.726+0000 7f6f50f90640 1 -- 172.21.15.111:0/350703576 wait complete. 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 Processor -- start 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 -- start start 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f380039e0 0x7f6f4c13a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c13d7b0 0x7f6f4c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 0x7f6f4c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c145b30 con 0x7f6f4c13dc30 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1459b0 con 0x7f6f380039e0 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f50f90640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c145cb0 con 0x7f6f4c13d7b0 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f4ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 0x7f6f4c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f4ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 0x7f6f4c0adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58908/0 (socket says 172.21.15.111:58908) 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f4ad9d640 1 -- 172.21.15.111:0/3684139492 learned_addr learned my addr 172.21.15.111:0/3684139492 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f380039e0 0x7f6f4c13a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.728+0000 7f6f41d9b640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c13d7b0 0x7f6f4c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f4ad9d640 1 -- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c13d7b0 msgr2=0x7f6f4c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f4ad9d640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c13d7b0 0x7f6f4c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f4ad9d640 1 -- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f380039e0 msgr2=0x7f6f4c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f4ad9d640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f380039e0 0x7f6f4c13a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f4ad9d640 1 -- 172.21.15.111:0/3684139492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c13e200 con 0x7f6f4c13dc30 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f4ad9d640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 0x7f6f4c0adfe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0aa870 tx=0x7f6f4400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44006b50 con 0x7f6f4c13dc30 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f44006d10 con 0x7f6f4c13dc30 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44003970 con 0x7f6f4c13dc30 2024-05-27T00:39:26.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c13e4e0 con 0x7f6f4c13dc30 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.729+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c13eaa0 con 0x7f6f4c13dc30 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.730+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c068680 con 0x7f6f4c13dc30 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.730+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f4401a020 con 0x7f6f4c13dc30 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.731+0000 7f6f437fe640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6f2c06be30 0x7f6f2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.731+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6f4408a5e0 con 0x7f6f4c13dc30 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.731+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6f2c06be30 0x7f6f2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:26.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.732+0000 7f6f4a59c640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6f2c06be30 0x7f6f2c06e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c033000 tx=0x7f6f3c035040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:26.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.735+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6f44058600 con 0x7f6f4c13dc30 2024-05-27T00:39:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.947+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f4c0b5ef0 con 0x7f6f4c13dc30 2024-05-27T00:39:26.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.947+0000 7f6f437fe640 1 -- 172.21.15.111:0/3684139492 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5186 (secure 0 0 0) 0x7f6f4405c900 con 0x7f6f4c13dc30 2024-05-27T00:39:26.950 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:26.950 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:25.959255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:39:26.950 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6f2c06be30 msgr2=0x7f6f2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6f2c06be30 0x7f6f2c06e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c033000 tx=0x7f6f3c035040 comp rx=0 tx=0).stop 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 msgr2=0x7f6f4c0adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 0x7f6f4c0adfe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0aa870 tx=0x7f6f4400eb90 comp rx=0 tx=0).stop 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 shutdown_connections 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6f2c06be30 0x7f6f2c06e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f4c13dc30 0x7f6f4c0adfe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6f4c13d7b0 0x7f6f4c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 --2- 172.21.15.111:0/3684139492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f380039e0 0x7f6f4c13a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:26.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 >> 172.21.15.111:0/3684139492 conn(0x7f6f4c073130 msgr2=0x7f6f4c143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:26.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 shutdown_connections 2024-05-27T00:39:26.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:26.950+0000 7f6f50f90640 1 -- 172.21.15.111:0/3684139492 wait complete. 2024-05-27T00:39:26.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:27.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.061+0000 7f3db03b7640 1 Processor -- start 2024-05-27T00:39:27.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.061+0000 7f3db03b7640 1 -- start start 2024-05-27T00:39:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3db03b7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da8071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3db03b7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3da806c330 0x7f3da80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3db03b7640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da8071fe0 con 0x7f3da8071b40 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3db03b7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f3da80722c0 con 0x7f3da806c330 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3db03b7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80725a0 con 0x7f3da8151740 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3da806c330 0x7f3da80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3da806c330 0x7f3da80cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58912/0 (socket says 172.21.15.111:58912) 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 -- 172.21.15.111:0/1929473888 learned_addr learned my addr 172.21.15.111:0/1929473888 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 -- 172.21.15.111:0/1929473888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 msgr2=0x7f3da8071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 --2- 172.21.15.111:0/1929473888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da8071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 -- 172.21.15.111:0/1929473888 >> v1:172.21.15.153:6789/0 conn(0x7f3da8071b40 legacy=0x7f3da806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.062+0000 7f3dae954640 1 -- 172.21.15.111:0/1929473888 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da8072880 con 0x7f3da806c330 2024-05-27T00:39:27.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.063+0000 7f3dae954640 1 --2- 172.21.15.111:0/1929473888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da806c330 0x7f3da80cf030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3da00098a0 tx=0x7f3da002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2d15ba4d28d011 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.063+0000 7f3dad151640 1 -- 172.21.15.111:0/1929473888 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0037070 con 0x7f3da806c330 2024-05-27T00:39:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.063+0000 7f3dad151640 1 -- 172.21.15.111:0/1929473888 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da0032440 con 0x7f3da806c330 2024-05-27T00:39:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.063+0000 7f3dad151640 1 -- 172.21.15.111:0/1929473888 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0032810 con 0x7f3da806c330 2024-05-27T00:39:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 -- 172.21.15.111:0/1929473888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da806c330 msgr2=0x7f3da80cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1929473888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da806c330 0x7f3da80cf030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3da00098a0 tx=0x7f3da002a250 comp rx=0 tx=0).stop 2024-05-27T00:39:27.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 -- 172.21.15.111:0/1929473888 shutdown_connections 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1929473888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da806c330 0x7f3da80cf030 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1929473888 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da8071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 -- 172.21.15.111:0/1929473888 >> 172.21.15.111:0/1929473888 conn(0x7f3da80b1c00 msgr2=0x7f3da80b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 -- 172.21.15.111:0/1929473888 shutdown_connections 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.064+0000 7f3db03b7640 1 -- 172.21.15.111:0/1929473888 wait complete. 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.065+0000 7f3db03b7640 1 Processor -- start 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.065+0000 7f3db03b7640 1 -- start start 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.065+0000 7f3db03b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da806c330 0x7f3da8149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.065+0000 7f3db03b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 0x7f3da8158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3db03b7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da80b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3db03b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3da815b090 con 0x7f3da8071b40 2024-05-27T00:39:27.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3db03b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3da815af10 con 0x7f3da806c330 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3db03b7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3da806b390 con 0x7f3da8151740 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 0x7f3da8158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 0x7f3da8158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58916/0 (socket says 172.21.15.111:58916) 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 -- 172.21.15.111:0/1116490065 learned_addr learned my addr 172.21.15.111:0/1116490065 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dae153640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da806c330 0x7f3da8149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dae954640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da80b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 -- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 msgr2=0x7f3da80b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da80b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 -- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da806c330 msgr2=0x7f3da8149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da806c330 0x7f3da8149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 -- 172.21.15.111:0/1116490065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da81550d0 con 0x7f3da8071b40 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3dad952640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 0x7f3da8158010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c410 tx=0x7f3d9800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98017070 con 0x7f3da8071b40 2024-05-27T00:39:27.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d980035f0 con 0x7f3da8071b40 2024-05-27T00:39:27.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da8156f70 con 0x7f3da8071b40 2024-05-27T00:39:27.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980039e0 con 0x7f3da8071b40 2024-05-27T00:39:27.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.066+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da8157530 con 0x7f3da8071b40 2024-05-27T00:39:27.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.067+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da806f510 con 0x7f3da8071b40 2024-05-27T00:39:27.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.068+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d9801a070 con 0x7f3da8071b40 2024-05-27T00:39:27.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.068+0000 7f3d9effd640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.068+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3d9808a560 con 0x7f3da8071b40 2024-05-27T00:39:27.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.068+0000 7f3dae153640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.069+0000 7f3dae153640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3da81569f0 tx=0x7f3da400d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.072+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3da806f510 con 0x7f3da8071b40 2024-05-27T00:39:27.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.247+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3da80cc620 con 0x7f3da8071b40 2024-05-27T00:39:27.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.248+0000 7f3d9effd640 1 -- 172.21.15.111:0/1116490065 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18575 (secure 0 0 0) 0x7f3d980010c0 con 0x7f3da8071b40 2024-05-27T00:39:27.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:27.250 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:25.959007+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:27.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3d8c06be30 msgr2=0x7f3d8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3da81569f0 tx=0x7f3da400d040 comp rx=0 tx=0).stop 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 msgr2=0x7f3da8158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 0x7f3da8158010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c410 tx=0x7f3d9800c8d0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 shutdown_connections 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3d8c06be30 0x7f3d8c06e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3da8151740 0x7f3da80b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3da8071b40 0x7f3da8158010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 --2- 172.21.15.111:0/1116490065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3da806c330 0x7f3da8149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 >> 172.21.15.111:0/1116490065 conn(0x7f3da80b1c00 msgr2=0x7f3da80b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 shutdown_connections 2024-05-27T00:39:27.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.250+0000 7f3db03b7640 1 -- 172.21.15.111:0/1116490065 wait complete. 2024-05-27T00:39:27.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:27.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.360+0000 7fd21b342640 1 Processor -- start 2024-05-27T00:39:27.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.361+0000 7fd21b342640 1 -- start start 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.361+0000 7fd21b342640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd214162050 0x7fd21415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd21b342640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd214162430 0x7fd2140af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd21b342640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2140af7a0 0x7fd2140b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd21b342640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd2140b4630 con 0x7fd214162430 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd21b342640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd2140afc50 con 0x7fd214162050 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd21b342640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd2140b3a20 con 0x7fd2140af7a0 2024-05-27T00:39:27.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd214162430 0x7fd2140af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd214162430 0x7fd2140af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58926/0 (socket says 172.21.15.111:58926) 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 -- 172.21.15.111:0/227754694 learned_addr learned my addr 172.21.15.111:0/227754694 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 -- 172.21.15.111:0/227754694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2140af7a0 msgr2=0x7fd2140b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 --2- 172.21.15.111:0/227754694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2140af7a0 0x7fd2140b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 -- 172.21.15.111:0/227754694 >> v2:172.21.15.153:3300/0 conn(0x7fd214162050 msgr2=0x7fd21415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2190de640 1 --2- 172.21.15.111:0/227754694 >> v2:172.21.15.153:3300/0 conn(0x7fd214162050 0x7fd21415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 --2- 172.21.15.111:0/227754694 >> v2:172.21.15.153:3300/0 conn(0x7fd214162050 0x7fd21415f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.362+0000 7fd2188dd640 1 -- 172.21.15.111:0/227754694 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2140b3d00 con 0x7fd214162430 2024-05-27T00:39:27.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd2188dd640 1 --2- 172.21.15.111:0/227754694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162430 0x7fd2140af260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd214072e30 tx=0x7fd20802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33901057a03e7b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd213fff640 1 -- 172.21.15.111:0/227754694 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208037070 con 0x7fd214162430 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd213fff640 1 -- 172.21.15.111:0/227754694 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208032440 con 0x7fd214162430 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd213fff640 1 -- 172.21.15.111:0/227754694 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208032810 con 0x7fd214162430 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 -- 172.21.15.111:0/227754694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162430 msgr2=0x7fd2140af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 --2- 172.21.15.111:0/227754694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162430 0x7fd2140af260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd214072e30 tx=0x7fd20802a1e0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 -- 172.21.15.111:0/227754694 shutdown_connections 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 --2- 172.21.15.111:0/227754694 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2140af7a0 0x7fd2140b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 --2- 172.21.15.111:0/227754694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162430 0x7fd2140af260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 --2- 172.21.15.111:0/227754694 >> v2:172.21.15.153:3300/0 conn(0x7fd214162050 0x7fd21415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.363+0000 7fd21b342640 1 -- 172.21.15.111:0/227754694 >> 172.21.15.111:0/227754694 conn(0x7fd214072a30 msgr2=0x7fd21406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.364+0000 7fd21b342640 1 -- 172.21.15.111:0/227754694 shutdown_connections 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.364+0000 7fd21b342640 1 -- 172.21.15.111:0/227754694 wait complete. 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 Processor -- start 2024-05-27T00:39:27.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 -- start start 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2140af7a0 0x7fd2140cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 0x7fd21414cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd214162430 0x7fd21406e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2140b5000 con 0x7fd214162050 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2140b42c0 con 0x7fd2140af7a0 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd21b342640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd2140b5180 con 0x7fd214162430 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd2188dd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 0x7fd21414cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.365+0000 7fd2188dd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 0x7fd21414cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58932/0 (socket says 172.21.15.111:58932) 2024-05-27T00:39:27.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 -- 172.21.15.111:0/3071096098 learned_addr learned my addr 172.21.15.111:0/3071096098 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2198df640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd214162430 0x7fd21406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 -- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd214162430 msgr2=0x7fd21406e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2190de640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2140af7a0 0x7fd2140cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd214162430 0x7fd21406e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 -- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2140af7a0 msgr2=0x7fd2140cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2140af7a0 0x7fd2140cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 -- 172.21.15.111:0/3071096098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd214154a30 con 0x7fd214162050 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd2188dd640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 0x7fd21414cf30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd2080090b0 tx=0x7fd2080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208037040 con 0x7fd214162050 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208042070 con 0x7fd214162050 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2140cc7a0 con 0x7fd214162050 2024-05-27T00:39:27.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208030bd0 con 0x7fd214162050 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.366+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21414e4c0 con 0x7fd214162050 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.367+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd214068680 con 0x7fd214162050 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.368+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd208047460 con 0x7fd214162050 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.368+0000 7fd211ffb640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd20006be30 0x7fd20006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.368+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd2080abe30 con 0x7fd214162050 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.368+0000 7fd2190de640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd20006be30 0x7fd20006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.369+0000 7fd2190de640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd20006be30 0x7fd20006e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd20c006fd0 tx=0x7fd20c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.373+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd208045360 con 0x7fd214162050 2024-05-27T00:39:27.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.547+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2140b10d0 con 0x7fd214162050 2024-05-27T00:39:27.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.548+0000 7fd211ffb640 1 -- 172.21.15.111:0/3071096098 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18575 (secure 0 0 0) 0x7fd20807a4a0 con 0x7fd214162050 2024-05-27T00:39:27.550 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:27.550 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":24,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:25.959007+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd20006be30 msgr2=0x7fd20006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd20006be30 0x7fd20006e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd20c006fd0 tx=0x7fd20c008040 comp rx=0 tx=0).stop 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 msgr2=0x7fd21414cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 0x7fd21414cf30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd2080090b0 tx=0x7fd2080046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 shutdown_connections 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd20006be30 0x7fd20006e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd214162430 0x7fd21406e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd214162050 0x7fd21414cf30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 --2- 172.21.15.111:0/3071096098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2140af7a0 0x7fd2140cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 >> 172.21.15.111:0/3071096098 conn(0x7fd214072a30 msgr2=0x7fd21406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:27.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 shutdown_connections 2024-05-27T00:39:27.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.550+0000 7fd21b342640 1 -- 172.21.15.111:0/3071096098 wait complete. 2024-05-27T00:39:27.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:27.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.682+0000 7f7ae429e640 1 Processor -- start 2024-05-27T00:39:27.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.682+0000 7f7ae429e640 1 -- start start 2024-05-27T00:39:27.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae429e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7adc06b390 0x7f7adc0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae429e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 0x7f7adc0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae429e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc0b36e0 con 0x7f7adc144000 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae429e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc0b42f0 con 0x7f7adc06b390 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae429e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc0b39c0 con 0x7f7adc0cdc80 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 0x7f7adc0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae1839640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7adc06b390 0x7f7adc0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 0x7f7adc0726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:35730/0 (socket says 172.21.15.111:35730) 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 -- 172.21.15.111:0/1834317179 learned_addr learned my addr 172.21.15.111:0/1834317179 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae203a640 1 --1- 172.21.15.111:0/1834317179 >> v1:172.21.15.153:6789/0 conn(0x7f7adc144000 0x7f7adc0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:54758/0 (socket says 172.21.15.111:54758) 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 -- 172.21.15.111:0/1834317179 >> v2:172.21.15.153:3300/0 conn(0x7f7adc06b390 msgr2=0x7f7adc0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 --2- 172.21.15.111:0/1834317179 >> v2:172.21.15.153:3300/0 conn(0x7f7adc06b390 0x7f7adc0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 -- 172.21.15.111:0/1834317179 >> v1:172.21.15.153:6789/0 conn(0x7f7adc144000 legacy=0x7f7adc0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.683+0000 7f7ae283b640 1 -- 172.21.15.111:0/1834317179 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc0b3ca0 con 0x7f7adc0cdc80 2024-05-27T00:39:27.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.684+0000 7f7ae1839640 1 --2- 172.21.15.111:0/1834317179 >> v2:172.21.15.153:3300/0 conn(0x7f7adc06b390 0x7f7adc0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:27.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.684+0000 7f7ae283b640 1 --2- 172.21.15.111:0/1834317179 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 0x7f7adc0726d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009e50 tx=0x7f7ad402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af50d5368a7b4646 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.684+0000 7f7ae1038640 1 -- 172.21.15.111:0/1834317179 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4036070 con 0x7f7adc0cdc80 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.684+0000 7f7ae1038640 1 -- 172.21.15.111:0/1834317179 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4031600 con 0x7f7adc0cdc80 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.684+0000 7f7ae1038640 1 -- 172.21.15.111:0/1834317179 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40319d0 con 0x7f7adc0cdc80 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 -- 172.21.15.111:0/1834317179 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 msgr2=0x7f7adc0726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1834317179 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 0x7f7adc0726d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009e50 tx=0x7f7ad402a710 comp rx=0 tx=0).stop 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 -- 172.21.15.111:0/1834317179 shutdown_connections 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1834317179 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc0cdc80 0x7f7adc0726d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1834317179 >> v2:172.21.15.153:3300/0 conn(0x7f7adc06b390 0x7f7adc0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 -- 172.21.15.111:0/1834317179 >> 172.21.15.111:0/1834317179 conn(0x7f7adc0b1c00 msgr2=0x7f7adc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 -- 172.21.15.111:0/1834317179 shutdown_connections 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.685+0000 7f7ae429e640 1 -- 172.21.15.111:0/1834317179 wait complete. 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.686+0000 7f7ae429e640 1 Processor -- start 2024-05-27T00:39:27.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.686+0000 7f7ae429e640 1 -- start start 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae429e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc06b390 0x7f7adc157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae429e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7adc0cdc80 0x7f7adc0abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae429e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 0x7f7adc0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae429e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc072d90 con 0x7f7adc144000 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae429e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc072c10 con 0x7f7adc0cdc80 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae429e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc072f10 con 0x7f7adc06b390 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 0x7f7adc0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 0x7f7adc0ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58948/0 (socket says 172.21.15.111:58948) 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 -- 172.21.15.111:0/1625520963 learned_addr learned my addr 172.21.15.111:0/1625520963 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae1839640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7adc0cdc80 0x7f7adc0abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae203a640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc06b390 0x7f7adc157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 -- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc06b390 msgr2=0x7f7adc157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc06b390 0x7f7adc157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 -- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7adc0cdc80 msgr2=0x7f7adc0abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7adc0cdc80 0x7f7adc0abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae283b640 1 -- 172.21.15.111:0/1625520963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc157760 con 0x7f7adc144000 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.687+0000 7f7ae203a640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc06b390 0x7f7adc157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.688+0000 7f7ae283b640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 0x7f7adc0ac140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009f70 tx=0x7f7ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.688+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4041020 con 0x7f7adc144000 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.688+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4007820 con 0x7f7adc144000 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.688+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc0aa1d0 con 0x7f7adc144000 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.688+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4036040 con 0x7f7adc144000 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.688+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc0aa790 con 0x7f7adc144000 2024-05-27T00:39:27.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.689+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc068680 con 0x7f7adc144000 2024-05-27T00:39:27.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.689+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ad4044020 con 0x7f7adc144000 2024-05-27T00:39:27.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.690+0000 7f7ad2ffd640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7ac006be30 0x7f7ac006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:27.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.690+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7ad40abe30 con 0x7f7adc144000 2024-05-27T00:39:27.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.690+0000 7f7ae203a640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7ac006be30 0x7f7ac006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:27.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.690+0000 7f7ae203a640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7ac006be30 0x7f7ac006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000f60 tx=0x7f7ad8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:27.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.694+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7ad404b030 con 0x7f7adc144000 2024-05-27T00:39:27.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.902+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7adc06bf60 con 0x7f7adc144000 2024-05-27T00:39:27.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.902+0000 7f7ad2ffd640 1 -- 172.21.15.111:0/1625520963 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5186 (secure 0 0 0) 0x7f7ad407a4b0 con 0x7f7adc144000 2024-05-27T00:39:27.905 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:25.959255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:39:27.905 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7ac006be30 msgr2=0x7f7ac006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7ac006be30 0x7f7ac006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000f60 tx=0x7f7ad8009040 comp rx=0 tx=0).stop 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 msgr2=0x7f7adc0ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 0x7f7adc0ac140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009f70 tx=0x7f7ad4004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 shutdown_connections 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7ac006be30 0x7f7ac006e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7adc144000 0x7f7adc0ac140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7adc0cdc80 0x7f7adc0abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.904+0000 7f7ae429e640 1 --2- 172.21.15.111:0/1625520963 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7adc06b390 0x7f7adc157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.905+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 >> 172.21.15.111:0/1625520963 conn(0x7f7adc0b1c00 msgr2=0x7f7adc0af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.905+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 shutdown_connections 2024-05-27T00:39:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:27.905+0000 7f7ae429e640 1 -- 172.21.15.111:0/1625520963 wait complete. 2024-05-27T00:39:27.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:28.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.016+0000 7f029bd9e640 1 Processor -- start 2024-05-27T00:39:28.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.016+0000 7f029bd9e640 1 -- start start 2024-05-27T00:39:28.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f029bd9e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f029406b390 0x7f02940cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f029bd9e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02940b36e0 con 0x7f02940cdc80 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f029bd9e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02940b39c0 con 0x7f0294144000 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f029bd9e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f02940b42f0 con 0x7f029406b390 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f029a33b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f02940cdc80 0x7f02940726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35378/0 (socket says 172.21.15.111:35378) 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f029a33b640 1 -- 172.21.15.111:0/2642680067 learned_addr learned my addr 172.21.15.111:0/2642680067 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0299339640 1 --2- 172.21.15.111:0/2642680067 >> v2:172.21.15.153:3300/0 conn(0x7f029406b390 0x7f02940cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 361081383 0 0) 0x7f02940b36e0 con 0x7f02940cdc80 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0280003520 con 0x7f02940cdc80 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1262888292 0 0) 0x7f0280003520 con 0x7f02940cdc80 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 >> v2:172.21.15.153:3300/0 conn(0x7f029406b390 msgr2=0x7f02940cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 --2- 172.21.15.111:0/2642680067 >> v2:172.21.15.153:3300/0 conn(0x7f029406b390 0x7f02940cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 >> v1:172.21.15.153:6789/0 conn(0x7f0294144000 legacy=0x7f02940b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02940b3ca0 con 0x7f02940cdc80 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0299339640 1 --2- 172.21.15.111:0/2642680067 >> v2:172.21.15.153:3300/0 conn(0x7f029406b390 0x7f02940cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f028c001670 con 0x7f02940cdc80 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 >> v1:172.21.15.111:6789/0 conn(0x7f02940cdc80 legacy=0x7f02940726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02800039b0 0x7f0280003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0280004800 0x7f0280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02800051c0 0x7f028000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02940b42f0 con 0x7f0280004800 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f028000c060 con 0x7f02800039b0 2024-05-27T00:39:28.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.017+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f028000c260 con 0x7f02800051c0 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0280004800 0x7f0280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299b3a640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02800039b0 0x7f0280003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f029a33b640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02800051c0 0x7f028000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 -- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02800051c0 msgr2=0x7f028000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02800051c0 0x7f028000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 -- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02800039b0 msgr2=0x7f0280003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02800039b0 0x7f0280003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 -- 172.21.15.111:0/2642680067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0280003520 con 0x7f0280004800 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f029a33b640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02800051c0 0x7f028000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:28.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f029a33b640 1 -- 172.21.15.111:0/2642680067 reap_dead start 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.018+0000 7f0299339640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0280004800 0x7f0280004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0284009c60 tx=0x7f0280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5278691674963f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284009860 con 0x7f0280004800 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0284004020 con 0x7f0280004800 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f0298b38640 1 -- 172.21.15.111:0/2642680067 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02840043f0 con 0x7f0280004800 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 -- 172.21.15.111:0/2642680067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0280004800 msgr2=0x7f0280004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0280004800 0x7f0280004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0284009c60 tx=0x7f0280005830 comp rx=0 tx=0).stop 2024-05-27T00:39:28.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 -- 172.21.15.111:0/2642680067 shutdown_connections 2024-05-27T00:39:28.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 --2- 172.21.15.111:0/2642680067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0280004800 0x7f0280004c80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 -- 172.21.15.111:0/2642680067 >> 172.21.15.111:0/2642680067 conn(0x7f02940b1c00 msgr2=0x7f02940b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 -- 172.21.15.111:0/2642680067 shutdown_connections 2024-05-27T00:39:28.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.019+0000 7f029bd9e640 1 -- 172.21.15.111:0/2642680067 wait complete. 2024-05-27T00:39:28.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.020+0000 7f029bd9e640 1 Processor -- start 2024-05-27T00:39:28.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.020+0000 7f029bd9e640 1 -- start start 2024-05-27T00:39:28.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029bd9e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0280004800 0x7f02940aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029bd9e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0294159bf0 0x7f02940a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029bd9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 0x7f02940cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029bd9e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02940cd8c0 con 0x7f02941593c0 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029bd9e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02940cd740 con 0x7f0294159bf0 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029bd9e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f02940cda40 con 0x7f0280004800 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 0x7f02940cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 0x7f02940cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58974/0 (socket says 172.21.15.111:58974) 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 -- 172.21.15.111:0/321794337 learned_addr learned my addr 172.21.15.111:0/321794337 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f0299b3a640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0280004800 0x7f02940aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f0299339640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0294159bf0 0x7f02940a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 -- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0280004800 msgr2=0x7f02940aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0280004800 0x7f02940aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 -- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0294159bf0 msgr2=0x7f02940a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0294159bf0 0x7f02940a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.021+0000 7f029a33b640 1 -- 172.21.15.111:0/321794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02940a9c30 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.022+0000 7f029a33b640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 0x7f02940cefd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f029406f710 tx=0x7f028c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.022+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00ebb0 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.022+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c019070 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.022+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02940a9f10 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.022+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c012450 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.022+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02940aa4d0 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.023+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0294068680 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.023+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f028c0125f0 con 0x7f02941593c0 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.024+0000 7f028a7fc640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f027806be30 0x7f027806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.024+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f028c08cdc0 con 0x7f02941593c0 2024-05-27T00:39:28.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.024+0000 7f0299b3a640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f027806be30 0x7f027806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.024+0000 7f0299b3a640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f027806be30 0x7f027806e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0290003ed0 tx=0x7f0290009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.028+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f028c05b430 con 0x7f02941593c0 2024-05-27T00:39:28.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.205+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02940ce7d0 con 0x7f02941593c0 2024-05-27T00:39:28.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.206+0000 7f028a7fc640 1 -- 172.21.15.111:0/321794337 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f028c05ad30 con 0x7f02941593c0 2024-05-27T00:39:28.208 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:28.209 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f027806be30 msgr2=0x7f027806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f027806be30 0x7f027806e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0290003ed0 tx=0x7f0290009040 comp rx=0 tx=0).stop 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 msgr2=0x7f02940cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 0x7f02940cefd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f029406f710 tx=0x7f028c010da0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 shutdown_connections 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f027806be30 0x7f027806e2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02941593c0 0x7f02940cefd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0294159bf0 0x7f02940a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 --2- 172.21.15.111:0/321794337 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0280004800 0x7f02940aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 >> 172.21.15.111:0/321794337 conn(0x7f02940b1c00 msgr2=0x7f0294072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 shutdown_connections 2024-05-27T00:39:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.208+0000 7f029bd9e640 1 -- 172.21.15.111:0/321794337 wait complete. 2024-05-27T00:39:28.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T00:39:20.908117+0000', 'modified': '2024-05-27T00:39:25.959255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4320}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4320': {'gid': 4320, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6836/2203621473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 2203621473}, {'type': 'v1', 'addr': '172.21.15.153:6836', 'nonce': 2203621473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4320, 'qdb_cluster': [4320]} 2024-05-27T00:39:28.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T00:39:28.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-27T00:39:28.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.726+0000 7fe8f82e1640 1 Processor -- start 2024-05-27T00:39:28.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.726+0000 7fe8f82e1640 1 -- start start 2024-05-27T00:39:28.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f82e1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 0x7fe8f00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f82e1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8f006b390 0x7fe8f00cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f82e1640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f00b3440 con 0x7fe8f00cdc80 2024-05-27T00:39:28.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f82e1640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe8f00b42f0 con 0x7fe8f006b390 2024-05-27T00:39:28.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f82e1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f00b36e0 con 0x7fe8f0144000 2024-05-27T00:39:28.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f607d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 0x7fe8f00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f587c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8f006b390 0x7fe8f00cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f607d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 0x7fe8f00b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:35756/0 (socket says 172.21.15.111:35756) 2024-05-27T00:39:28.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f607d640 1 -- 172.21.15.111:0/2195977604 learned_addr learned my addr 172.21.15.111:0/2195977604 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.727+0000 7fe8f687e640 1 --1- 172.21.15.111:0/2195977604 >> v1:172.21.15.153:6789/0 conn(0x7fe8f00cdc80 0x7fe8f00726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:54778/0 (socket says 172.21.15.111:54778) 2024-05-27T00:39:28.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.728+0000 7fe8f607d640 1 -- 172.21.15.111:0/2195977604 >> v2:172.21.15.153:3300/0 conn(0x7fe8f006b390 msgr2=0x7fe8f00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.728+0000 7fe8f607d640 1 --2- 172.21.15.111:0/2195977604 >> v2:172.21.15.153:3300/0 conn(0x7fe8f006b390 0x7fe8f00cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.728+0000 7fe8f607d640 1 -- 172.21.15.111:0/2195977604 >> v1:172.21.15.153:6789/0 conn(0x7fe8f00cdc80 legacy=0x7fe8f00726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.728+0000 7fe8f607d640 1 -- 172.21.15.111:0/2195977604 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f00b39c0 con 0x7fe8f0144000 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.728+0000 7fe8f587c640 1 --2- 172.21.15.111:0/2195977604 >> v2:172.21.15.153:3300/0 conn(0x7fe8f006b390 0x7fe8f00cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.728+0000 7fe8f607d640 1 --2- 172.21.15.111:0/2195977604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 0x7fe8f00b2f00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009e50 tx=0x7fe8e002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d909305873f1ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f507b640 1 -- 172.21.15.111:0/2195977604 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0036070 con 0x7fe8f0144000 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f507b640 1 -- 172.21.15.111:0/2195977604 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e0031600 con 0x7fe8f0144000 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f507b640 1 -- 172.21.15.111:0/2195977604 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00319d0 con 0x7fe8f0144000 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 -- 172.21.15.111:0/2195977604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 msgr2=0x7fe8f00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/2195977604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 0x7fe8f00b2f00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009e50 tx=0x7fe8e002a710 comp rx=0 tx=0).stop 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 -- 172.21.15.111:0/2195977604 shutdown_connections 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/2195977604 >> v2:172.21.15.153:3300/0 conn(0x7fe8f006b390 0x7fe8f00cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/2195977604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f0144000 0x7fe8f00b2f00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 -- 172.21.15.111:0/2195977604 >> 172.21.15.111:0/2195977604 conn(0x7fe8f00b1c00 msgr2=0x7fe8f00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 -- 172.21.15.111:0/2195977604 shutdown_connections 2024-05-27T00:39:28.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.729+0000 7fe8f82e1640 1 -- 172.21.15.111:0/2195977604 wait complete. 2024-05-27T00:39:28.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.730+0000 7fe8f82e1640 1 Processor -- start 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 -- start start 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8f006b390 0x7fe8f013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f00cdc80 0x7fe8f0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 0x7fe8f0159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0072de0 con 0x7fe8f0144000 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f00b3f80 con 0x7fe8f006b390 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f82e1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f0072f60 con 0x7fe8f00cdc80 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f687e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 0x7fe8f0159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f687e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 0x7fe8f0159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58986/0 (socket says 172.21.15.111:58986) 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.731+0000 7fe8f687e640 1 -- 172.21.15.111:0/1485711959 learned_addr learned my addr 172.21.15.111:0/1485711959 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f587c640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f00cdc80 0x7fe8f0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8f006b390 0x7fe8f013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f687e640 1 -- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f00cdc80 msgr2=0x7fe8f0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f687e640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f00cdc80 0x7fe8f0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f687e640 1 -- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8f006b390 msgr2=0x7fe8f013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f687e640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8f006b390 0x7fe8f013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f687e640 1 -- 172.21.15.111:0/1485711959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f0159b10 con 0x7fe8f0144000 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8f006b390 0x7fe8f013f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f687e640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 0x7fe8f0159400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c430 tx=0x7fe8e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8eeffd640 1 -- 172.21.15.111:0/1485711959 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80074e0 con 0x7fe8f0144000 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8eeffd640 1 -- 172.21.15.111:0/1485711959 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8007680 con 0x7fe8f0144000 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f0159df0 con 0x7fe8f0144000 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8eeffd640 1 -- 172.21.15.111:0/1485711959 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80049e0 con 0x7fe8f0144000 2024-05-27T00:39:28.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.732+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f00a9400 con 0x7fe8f0144000 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.733+0000 7fe8eeffd640 1 -- 172.21.15.111:0/1485711959 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8e801a020 con 0x7fe8f0144000 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.734+0000 7fe8eeffd640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe8dc06be30 0x7fe8dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.734+0000 7fe8eeffd640 1 -- 172.21.15.111:0/1485711959 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8e808a920 con 0x7fe8f0144000 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.734+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe8dc06be30 0x7fe8dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.735+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe8dc06be30 0x7fe8dc06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0033000 tx=0x7fe8e0009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.735+0000 7fe8faff2740 1 Processor -- start 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.735+0000 7fe8faff2740 1 -- start start 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e869510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e869a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e839960 0x55ab9e83d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8faff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ab9e7a2740 con 0x55ab9e866d00 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab9e740a80 con 0x55ab9e839960 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55ab9e79e700 con 0x55ab9e8670e0 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e869510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e869510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58994/0 (socket says 172.21.15.111:58994) 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 -- 172.21.15.111:0/942711874 learned_addr learned my addr 172.21.15.111:0/942711874 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f587c640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e869a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f687e640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e839960 0x55ab9e83d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 -- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 msgr2=0x55ab9e869a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e869a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 -- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e839960 msgr2=0x55ab9e83d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e839960 0x55ab9e83d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 -- 172.21.15.111:0/942711874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9e76bfb0 con 0x55ab9e866d00 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f587c640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e869a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f687e640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e839960 0x55ab9e83d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8f607d640 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e869510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009c90 tx=0x7fe8e002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b5302bf608e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.736+0000 7fe8d7fff640 1 -- 172.21.15.111:0/942711874 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0036040 con 0x55ab9e866d00 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8d7fff640 1 -- 172.21.15.111:0/942711874 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e004c070 con 0x55ab9e866d00 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8d7fff640 1 -- 172.21.15.111:0/942711874 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e003cd60 con 0x55ab9e866d00 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 -- 172.21.15.111:0/942711874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 msgr2=0x55ab9e869510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e869510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009c90 tx=0x7fe8e002e000 comp rx=0 tx=0).stop 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 -- 172.21.15.111:0/942711874 shutdown_connections 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e839960 0x55ab9e83d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e869a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 --2- 172.21.15.111:0/942711874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e869510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 -- 172.21.15.111:0/942711874 >> 172.21.15.111:0/942711874 conn(0x55ab9e83bbd0 msgr2=0x55ab9e884510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 -- 172.21.15.111:0/942711874 shutdown_connections 2024-05-27T00:39:28.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.737+0000 7fe8faff2740 1 -- 172.21.15.111:0/942711874 wait complete. 2024-05-27T00:39:28.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.738+0000 7fe8faff2740 1 -- start start 2024-05-27T00:39:28.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.738+0000 7fe8faff2740 1 Processor -- start 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e8892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e887e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e8869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ab9e83ed80 con 0x55ab9e86c6d0 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab9e72bc30 con 0x55ab9e8670e0 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55ab9e83f060 con 0x55ab9e866d00 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 10 client.0 _ll_register_callbacks cb 0x55ab9e75d7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e8869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e8869a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58996/0 (socket says 172.21.15.111:58996) 2024-05-27T00:39:28.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 -- 172.21.15.111:0/1529822388 learned_addr learned my addr 172.21.15.111:0/1529822388 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 -- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 msgr2=0x55ab9e8892a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e8892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f587c640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e887e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e8892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 -- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 msgr2=0x55ab9e887e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e887e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 -- 172.21.15.111:0/1529822388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9e4b62d0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f587c640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e887e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8f687e640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e8869a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8060f40 tx=0x7fe8e805cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa9466833f7ebd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8d57fa640 1 -- 172.21.15.111:0/1529822388 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8058cf0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8d57fa640 1 -- 172.21.15.111:0/1529822388 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8e8060fe0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.739+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e004c070 con 0x55ab9e86c6d0 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8d57fa640 1 -- 172.21.15.111:0/1529822388 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8056c50 con 0x55ab9e86c6d0 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 10 client.4446 fetch_fsmap learned FSMap version 7 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ab9e5806b0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8d57fa640 1 -- 172.21.15.111:0/1529822388 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e004c070 con 0x55ab9e86c6d0 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8d57fa640 1 -- 172.21.15.111:0/1529822388 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fe8e8059bc0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 10 client.4446 fetch_fsmap finished waiting for FSMap version 7 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 10 client.4446 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 20 client.4446 populate_metadata read hostname 'smithi111' 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e898240 0x55ab9e898620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 4 client.4446 mds_command: new command op to 4320 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 --> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab9e7afe70 con 0x55ab9e898240 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e898240 0x55ab9e898620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8d57fa640 10 client.4446 ms_handle_connect on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.740+0000 7fe8f607d640 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e898240 0x55ab9e898620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1a31cc1d10234 server_cookie=e674101981cd7265 in_seq=0 out_seq=0 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8d57fa640 1 -- 172.21.15.111:0/1529822388 <== mds.0 v2:172.21.15.153:6834/2203621473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab9e7afe70 con 0x55ab9e898240 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8d57fa640 10 client.4446 handle_command_reply: tid=0 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 client.4446 shutdown 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 msgr2=0x55ab9e8869a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e8869a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8060f40 tx=0x7fe8e805cc40 comp rx=0 tx=0).stop 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 shutdown_connections 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e898240 0x55ab9e898620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e8869a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e887e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8d57fa640 0 client.4446 ms_handle_reset on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 --2- 172.21.15.111:0/1529822388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e8892a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.741+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 >> 172.21.15.111:0/1529822388 conn(0x55ab9e839960 msgr2=0x55ab9e865f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.742+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 shutdown_connections 2024-05-27T00:39:28.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.742+0000 7fe8faff2740 1 -- 172.21.15.111:0/1529822388 wait complete. 2024-05-27T00:39:28.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.742+0000 7fe8faff2740 20 client.4446 trim_cache size 0 max 16384 2024-05-27T00:39:28.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.760+0000 7fe8faff2740 1 Processor -- start 2024-05-27T00:39:28.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.760+0000 7fe8faff2740 1 -- start start 2024-05-27T00:39:28.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e89e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e88c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e86c6d0 0x55ab9e88c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8faff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0200d80 con 0x55ab9e866d00 2024-05-27T00:39:28.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f00aaa80 con 0x55ab9e86c6d0 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f0200730 con 0x55ab9e8670e0 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e89e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e89e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59010/0 (socket says 172.21.15.111:59010) 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 -- 172.21.15.111:0/4138858836 learned_addr learned my addr 172.21.15.111:0/4138858836 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 -- 172.21.15.111:0/4138858836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 msgr2=0x55ab9e88c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e88c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 -- 172.21.15.111:0/4138858836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e86c6d0 msgr2=0x55ab9e88c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e86c6d0 0x55ab9e88c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 -- 172.21.15.111:0/4138858836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9e885690 con 0x55ab9e866d00 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8f607d640 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e89e650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0000f80 tx=0x7fe8e003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f633cac7abd89dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8d77fe640 1 -- 172.21.15.111:0/4138858836 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0036040 con 0x55ab9e866d00 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8d77fe640 1 -- 172.21.15.111:0/4138858836 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e0054070 con 0x55ab9e866d00 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.761+0000 7fe8d77fe640 1 -- 172.21.15.111:0/4138858836 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0056310 con 0x55ab9e866d00 2024-05-27T00:39:28.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 -- 172.21.15.111:0/4138858836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 msgr2=0x55ab9e89e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e89e650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0000f80 tx=0x7fe8e003c000 comp rx=0 tx=0).stop 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 -- 172.21.15.111:0/4138858836 shutdown_connections 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e86c6d0 0x55ab9e88c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e8670e0 0x55ab9e88c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 --2- 172.21.15.111:0/4138858836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e866d00 0x55ab9e89e650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 -- 172.21.15.111:0/4138858836 >> 172.21.15.111:0/4138858836 conn(0x55ab9e839960 msgr2=0x55ab9e835100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 -- 172.21.15.111:0/4138858836 shutdown_connections 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 -- 172.21.15.111:0/4138858836 wait complete. 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 -- start start 2024-05-27T00:39:28.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.762+0000 7fe8faff2740 1 Processor -- start 2024-05-27T00:39:28.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e83a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e85d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e85d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ab9e889a90 con 0x55ab9e86c6d0 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55ab9e8861c0 con 0x55ab9e8670e0 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55ab9e86ccd0 con 0x55ab9e866d00 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8faff2740 10 client.0 _ll_register_callbacks cb 0x7fe8e8059040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8f687e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e85d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8f687e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e85d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59018/0 (socket says 172.21.15.111:59018) 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.763+0000 7fe8f687e640 1 -- 172.21.15.111:0/3831818523 learned_addr learned my addr 172.21.15.111:0/3831818523 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f607d640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e83a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f587c640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e85d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f687e640 1 -- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 msgr2=0x55ab9e83a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f687e640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e83a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f687e640 1 -- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 msgr2=0x55ab9e85d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f687e640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e85d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f687e640 1 -- 172.21.15.111:0/3831818523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9e8a8020 con 0x55ab9e86c6d0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f607d640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e83a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f587c640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e85d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8f687e640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e85d8b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800cdf0 tx=0x7fe8e80889e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b8ff22e009b075 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8d57fa640 1 -- 172.21.15.111:0/3831818523 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8058cf0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8d57fa640 1 -- 172.21.15.111:0/3831818523 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8e8004b80 con 0x55ab9e86c6d0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e0054070 con 0x55ab9e86c6d0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8d57fa640 1 -- 172.21.15.111:0/3831818523 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8004da0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 10 client.4452 fetch_fsmap learned FSMap version 7 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55ab9e865220 con 0x55ab9e86c6d0 2024-05-27T00:39:28.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8d57fa640 1 -- 172.21.15.111:0/3831818523 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e0054070 con 0x55ab9e86c6d0 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8d57fa640 1 -- 172.21.15.111:0/3831818523 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fe8e8057ce0 con 0x55ab9e86c6d0 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 10 client.4452 fetch_fsmap finished waiting for FSMap version 7 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 10 client.4452 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 20 client.4452 populate_metadata read hostname 'smithi111' 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e903d20 0x55ab9e83d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 4 client.4452 mds_command: new command op to 4320 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.764+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 --> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] -- command(tid 0: {"prefix": "status"}) -- 0x55ab9e7aa1c0 con 0x55ab9e903d20 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8f607d640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e903d20 0x55ab9e83d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8d57fa640 10 client.4452 ms_handle_connect on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8f607d640 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e903d20 0x55ab9e83d170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d274822a74639ae server_cookie=27813b800cce5ed2 in_seq=0 out_seq=0 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8d57fa640 1 -- 172.21.15.111:0/3831818523 <== mds.0 v2:172.21.15.153:6834/2203621473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55ab9e7aa1c0 con 0x55ab9e903d20 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8d57fa640 10 client.4452 handle_command_reply: tid=0 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 client.4452 shutdown 2024-05-27T00:39:28.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 msgr2=0x55ab9e85d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e85d8b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800cdf0 tx=0x7fe8e80889e0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 shutdown_connections 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x55ab9e903d20 0x55ab9e83d170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ab9e86c6d0 0x55ab9e85d8b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ab9e8670e0 0x55ab9e85d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8d57fa640 0 client.4452 ms_handle_reset on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 --2- 172.21.15.111:0/3831818523 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55ab9e866d00 0x55ab9e83a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 >> 172.21.15.111:0/3831818523 conn(0x55ab9e839960 msgr2=0x55ab9e8863a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 shutdown_connections 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 1 -- 172.21.15.111:0/3831818523 wait complete. 2024-05-27T00:39:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.765+0000 7fe8faff2740 20 client.4452 trim_cache size 0 max 16384 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "ef97d1dc-a8f7-4487-922a-ccc176f6927a", 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4320, 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 7.1880267179999997, 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 7, 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 25, 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 24, 2024-05-27T00:39:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 14.285470206999999 2024-05-27T00:39:28.776 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe8dc06be30 msgr2=0x7fe8dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe8dc06be30 0x7fe8dc06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0033000 tx=0x7fe8e0009000 comp rx=0 tx=0).stop 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 msgr2=0x7fe8f0159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 0x7fe8f0159400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c430 tx=0x7fe8e800c8f0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 shutdown_connections 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe8dc06be30 0x7fe8dc06e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f0144000 0x7fe8f0159400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe8f00cdc80 0x7fe8f0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 --2- 172.21.15.111:0/1485711959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8f006b390 0x7fe8f013f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 >> 172.21.15.111:0/1485711959 conn(0x7fe8f00b1c00 msgr2=0x7fe8f0143120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 shutdown_connections 2024-05-27T00:39:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.766+0000 7fe8f82e1640 1 -- 172.21.15.111:0/1485711959 wait complete. 2024-05-27T00:39:28.779 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T00:39:28.779 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:28.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T00:39:28.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 Processor -- start 2024-05-27T00:39:28.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 -- start start 2024-05-27T00:39:28.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa4300b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4300b77e0 0x7fa430161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa430162390 0x7fa4300d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa43006cc70 con 0x7fa430162390 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa4300d1e50 con 0x7fa4300b77e0 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa435b9e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa4300d2130 con 0x7fa4300b3630 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.878+0000 7fa42ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa430162390 0x7fa4300d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa430162390 0x7fa4300d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59034/0 (socket says 172.21.15.111:59034) 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 -- 172.21.15.111:0/2169497619 learned_addr learned my addr 172.21.15.111:0/2169497619 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42f7fe640 1 --2- 172.21.15.111:0/2169497619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa4300b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 -- 172.21.15.111:0/2169497619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 msgr2=0x7fa4300b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 --2- 172.21.15.111:0/2169497619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa4300b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 -- 172.21.15.111:0/2169497619 >> v2:172.21.15.153:3300/0 conn(0x7fa4300b77e0 msgr2=0x7fa430161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa426ffd640 1 --2- 172.21.15.111:0/2169497619 >> v2:172.21.15.153:3300/0 conn(0x7fa4300b77e0 0x7fa430161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 --2- 172.21.15.111:0/2169497619 >> v2:172.21.15.153:3300/0 conn(0x7fa4300b77e0 0x7fa430161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 -- 172.21.15.111:0/2169497619 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4300d2410 con 0x7fa430162390 2024-05-27T00:39:28.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.879+0000 7fa42ffff640 1 --2- 172.21.15.111:0/2169497619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa430162390 0x7fa4300d2c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa43008cd80 tx=0x7fa42802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3e2b7aba3c576e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa42effd640 1 -- 172.21.15.111:0/2169497619 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428036070 con 0x7fa430162390 2024-05-27T00:39:28.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa42effd640 1 -- 172.21.15.111:0/2169497619 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa428031600 con 0x7fa430162390 2024-05-27T00:39:28.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa42effd640 1 -- 172.21.15.111:0/2169497619 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4280319d0 con 0x7fa430162390 2024-05-27T00:39:28.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 -- 172.21.15.111:0/2169497619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa430162390 msgr2=0x7fa4300d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2169497619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa430162390 0x7fa4300d2c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa43008cd80 tx=0x7fa42802a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 -- 172.21.15.111:0/2169497619 shutdown_connections 2024-05-27T00:39:28.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2169497619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa430162390 0x7fa4300d2c10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2169497619 >> v2:172.21.15.153:3300/0 conn(0x7fa4300b77e0 0x7fa430161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2169497619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa4300b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 -- 172.21.15.111:0/2169497619 >> 172.21.15.111:0/2169497619 conn(0x7fa4300d42c0 msgr2=0x7fa4300d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 -- 172.21.15.111:0/2169497619 shutdown_connections 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.880+0000 7fa435b9e640 1 -- 172.21.15.111:0/2169497619 wait complete. 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.882+0000 7fa435b9e640 1 Processor -- start 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.882+0000 7fa435b9e640 1 -- start start 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa430151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 0x7fa43014ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa430162390 0x7fa43014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4300d32d0 con 0x7fa4300b77e0 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa4300d3150 con 0x7fa430162390 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa4300d3450 con 0x7fa4300b3630 2024-05-27T00:39:28.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 0x7fa43014ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 0x7fa43014ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59042/0 (socket says 172.21.15.111:59042) 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 -- 172.21.15.111:0/2349763512 learned_addr learned my addr 172.21.15.111:0/2349763512 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa42f7fe640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa430151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa42ffff640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa430162390 0x7fa43014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 -- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 msgr2=0x7fa430151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa430151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 -- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa430162390 msgr2=0x7fa43014f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa430162390 0x7fa43014f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 -- 172.21.15.111:0/2349763512 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4300aabb0 con 0x7fa4300b77e0 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa42f7fe640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa430151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa426ffd640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 0x7fa43014ee70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c7f0 tx=0x7fa41800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418015070 con 0x7fa4300b77e0 2024-05-27T00:39:28.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa418004240 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418007620 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4301523a0 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.883+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa430152960 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.884+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa430068680 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.885+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4180079d0 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.885+0000 7fa42cff9640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa41006be30 0x7fa41006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.885+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa41808b460 con 0x7fa4300b77e0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.885+0000 7fa42f7fe640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa41006be30 0x7fa41006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:28.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.886+0000 7fa42f7fe640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa41006be30 0x7fa41006e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa42000aa50 tx=0x7fa420008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:28.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:28.889+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa418059510 con 0x7fa4300b77e0 2024-05-27T00:39:29.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.104+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa430074c80 con 0x7fa4300b77e0 2024-05-27T00:39:29.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.105+0000 7fa42cff9640 1 -- 172.21.15.111:0/2349763512 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fa41805d810 con 0x7fa4300b77e0 2024-05-27T00:39:29.107 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa41006be30 msgr2=0x7fa41006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa41006be30 0x7fa41006e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa42000aa50 tx=0x7fa420008040 comp rx=0 tx=0).stop 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 msgr2=0x7fa43014ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 0x7fa43014ee70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c7f0 tx=0x7fa41800ccb0 comp rx=0 tx=0).stop 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 shutdown_connections 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa41006be30 0x7fa41006e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:29.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa430162390 0x7fa43014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:29.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4300b77e0 0x7fa43014ee70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:29.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 --2- 172.21.15.111:0/2349763512 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa4300b3630 0x7fa430151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:29.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 >> 172.21.15.111:0/2349763512 conn(0x7fa4300d42c0 msgr2=0x7fa4300b9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:29.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 shutdown_connections 2024-05-27T00:39:29.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29.107+0000 7fa435b9e640 1 -- 172.21.15.111:0/2349763512 wait complete. 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T00:39:29.120 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T00:39:29.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:29.121 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:29.136 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42751sec preferred_lft 42751sec 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-05-27T00:39:29.137 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:29.138 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T00:39:29.138 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:29.138 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:29.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-05-27T00:39:29.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-05-27T00:39:29.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-05-27T00:39:29.139 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T00:39:29.139 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:29.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:29.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:29.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:29.334 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T00:39:29.359 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-27T00:39:29.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:29.360 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-05-27T00:39:29.428 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-05-27T00:39:29.428 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T00:39:29.429 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T00:39:29.429 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T00:39:29.429 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T00:39:29.429 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:29.429 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:29.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T00:39:29.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T00:39:29.430 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T00:39:29.430 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:29.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:29.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:29.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:29.709 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-27T00:39:29.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:29.725 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-05-27T00:39:29.780 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:29.780 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:29.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:29.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-27T00:39:29.780 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:29.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:29.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:29.897 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-27T00:39:29.898 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:29.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:30.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:30.021 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:30.022 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:30.022 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-05-27T00:39:30.022 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T00:39:30.022 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:30.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:30.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:30.098 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-27T00:39:30.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:30.099 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:30.113 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T00:39:30.114 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:30.114 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:30.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-27T00:39:30.234 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T00:39:30.290 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-27T00:39:30.290 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-27T00:39:30.290 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-27T00:39:30.291 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T00:39:30.292 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-27T00:39:30.293 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-27T00:39:30.293 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2024-05-27T00:39:30.293 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2024-05-27T00:39:30.293 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2024-05-27T00:39:30.293 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2024-05-27T00:39:30.293 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-27T00:39:30.294 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:30.294 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:30.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:30.294 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-27T00:39:30.346 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T00:39:30.346 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-27T00:39:30.388 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-27T00:39:30.412 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-27T00:39:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-27T00:39:30.470 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:30.471 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:30.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:30.472 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-27T00:39:30.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:2024-05-27T00:39:30.521+0000 7f8a4f80c480 -1 init, newargv = 0x559c7ab28ea0 newargc=15 2024-05-27T00:39:30.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[44991]: starting ceph client 2024-05-27T00:39:30.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[44991]: starting fuse 2024-05-27T00:39:31.626 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2024-05-27T00:39:31.658 DEBUG:teuthology.orchestra.run.smithi111:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T00:39:31.717 INFO:teuthology.orchestra.run.smithi111.stdout:/proc 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/sys 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/dev 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/security 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/shm 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/pts 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/run 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/cgroup 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/pstore 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/bpf 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/ 2024-05-27T00:39:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/selinux 2024-05-27T00:39:31.719 INFO:teuthology.orchestra.run.smithi111.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T00:39:31.719 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/hugepages 2024-05-27T00:39:31.719 INFO:teuthology.orchestra.run.smithi111.stdout:/dev/mqueue 2024-05-27T00:39:31.719 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/debug 2024-05-27T00:39:31.719 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/tracing 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/fs/fuse/connections 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/sys/kernel/config 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/run/user/1000 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-0 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-1 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-2 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/var/lib/ceph/osd/ceph-3 2024-05-27T00:39:31.720 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns 2024-05-27T00:39:31.721 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:31.721 INFO:teuthology.orchestra.run.smithi111.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:31.721 INFO:teuthology.orchestra.run.smithi111.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:31.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:31.721 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2024-05-27T00:39:31.776 INFO:teuthology.orchestra.run.smithi111.stdout:40 2024-05-27T00:39:31.776 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-27T00:39:31.776 DEBUG:teuthology.orchestra.run.smithi111:> sudo stdin-killer -- python3 -c ' 2024-05-27T00:39:31.777 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2024-05-27T00:39:31.777 DEBUG:teuthology.orchestra.run.smithi111:> import re 2024-05-27T00:39:31.777 DEBUG:teuthology.orchestra.run.smithi111:> import os 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-27T00:39:31.778 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-27T00:39:31.779 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2024-05-27T00:39:31.779 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2024-05-27T00:39:31.779 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2024-05-27T00:39:31.779 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2024-05-27T00:39:31.779 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-27T00:39:31.779 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> return f 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.admin")) 2024-05-27T00:39:31.780 DEBUG:teuthology.orchestra.run.smithi111:> ' 2024-05-27T00:39:31.900 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.admin.44991.asok 2024-05-27T00:39:31.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:31.911 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44991.asok 2024-05-27T00:39:31.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:31.911 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44991.asok status 2024-05-27T00:39:32.046 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "0ef46a246e1118e5adafa1857b4189e7d780c816", 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc)", 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "admin", 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "44991", 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2024-05-27T00:39:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4464, 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4464 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 623953268 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-05-27T00:39:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 623953268 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4464 192.168.0.1:0/623953268", 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/623953268", 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 7, 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 25, 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2024-05-27T00:39:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2024-05-27T00:39:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2024-05-27T00:39:32.050 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-27T00:39:32.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:32.055 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2024-05-27T00:39:32.055 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:32.071 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2024-05-27T00:39:32.071 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:32.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:32.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:32.137 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-27T00:39:32.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:32.253 INFO:teuthology.orchestra.run.smithi111.stdout:# file: . 2024-05-27T00:39:32.253 INFO:teuthology.orchestra.run.smithi111.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-27T00:39:32.253 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:32.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:32.259 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:39:32.259 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:39:32.275 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:32.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.414+0000 7f1ad754f640 1 Processor -- start 2024-05-27T00:39:32.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.415+0000 7f1ad754f640 1 -- start start 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.415+0000 7f1ad754f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 0x7f1ad00b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad754f640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1ad0162560 0x7f1ad00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad754f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad006cc70 con 0x7f1ad00b77e0 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad754f640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad00d1e50 con 0x7f1ad0162560 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad754f640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad00d2130 con 0x7f1ad00b3630 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad4aea640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1ad00b77e0 0x7f1ad0161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35340/0 (socket says 172.21.15.111:35340) 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad4aea640 1 -- 172.21.15.111:0/2142067293 learned_addr learned my addr 172.21.15.111:0/2142067293 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637746944 0 0) 0x7f1ad006cc70 con 0x7f1ad00b77e0 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad5aec640 1 --2- 172.21.15.111:0/2142067293 >> v2:172.21.15.153:3300/0 conn(0x7f1ad0162560 0x7f1ad00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1abc003520 con 0x7f1ad00b77e0 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad52eb640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 0x7f1ad00b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1886679994 0 0) 0x7f1abc003520 con 0x7f1ad00b77e0 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 msgr2=0x7f1ad00b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 0x7f1ad00b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 >> v2:172.21.15.153:3300/0 conn(0x7f1ad0162560 msgr2=0x7f1ad00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/2142067293 >> v2:172.21.15.153:3300/0 conn(0x7f1ad0162560 0x7f1ad00d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad00d2410 con 0x7f1ad00b77e0 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f1ac4001670 con 0x7f1ad00b77e0 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 >> v1:172.21.15.111:6789/0 conn(0x7f1ad00b77e0 legacy=0x7f1ad0161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 0x7f1abc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc003f30 0x7f1abc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1abc004d40 0x7f1abc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00d1e50 con 0x7f1abc003f30 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00d2130 con 0x7f1abc004d40 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc00b9c0 con 0x7f1ad00b3630 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc003f30 0x7f1abc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad52eb640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 0x7f1abc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.416+0000 7f1ad5aec640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1abc004d40 0x7f1abc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ad4aea640 1 -- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 msgr2=0x7f1abc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ad00b3630 0x7f1abc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ad4aea640 1 -- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1abc004d40 msgr2=0x7f1abc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1abc004d40 0x7f1abc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ad4aea640 1 -- 172.21.15.111:0/2142067293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc003520 con 0x7f1abc003f30 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc003f30 0x7f1abc004800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4001050 tx=0x7f1abc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311ef3eb89c4ba42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4007d70 con 0x7f1abc003f30 2024-05-27T00:39:32.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac4005af0 con 0x7f1abc003f30 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.417+0000 7f1ac3fff640 1 -- 172.21.15.111:0/2142067293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4007070 con 0x7f1abc003f30 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad754f640 1 -- 172.21.15.111:0/2142067293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc003f30 msgr2=0x7f1abc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad754f640 1 --2- 172.21.15.111:0/2142067293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1abc003f30 0x7f1abc004800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4001050 tx=0x7f1abc005830 comp rx=0 tx=0).stop 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad5aec640 1 -- 172.21.15.111:0/2142067293 reap_dead start 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad754f640 1 -- 172.21.15.111:0/2142067293 shutdown_connections 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad754f640 1 -- 172.21.15.111:0/2142067293 >> 172.21.15.111:0/2142067293 conn(0x7f1ad00d42c0 msgr2=0x7f1ad00d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad754f640 1 -- 172.21.15.111:0/2142067293 shutdown_connections 2024-05-27T00:39:32.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.418+0000 7f1ad754f640 1 -- 172.21.15.111:0/2142067293 wait complete. 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.419+0000 7f1ad754f640 1 Processor -- start 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.419+0000 7f1ad754f640 1 -- start start 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad754f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1abc003f30 0x7f1ad00cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad754f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 0x7f1ad00cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad754f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad0156150 0x7f1ad01f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad754f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00b7420 con 0x7f1ad0156b80 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad754f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00b72a0 con 0x7f1ad0156150 2024-05-27T00:39:32.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad754f640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad00b75a0 con 0x7f1abc003f30 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 0x7f1ad00cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 0x7f1ad00cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56554/0 (socket says 172.21.15.111:56554) 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 -- 172.21.15.111:0/3772329508 learned_addr learned my addr 172.21.15.111:0/3772329508 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad52eb640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1abc003f30 0x7f1ad00cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad5aec640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad0156150 0x7f1ad01f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 -- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1abc003f30 msgr2=0x7f1ad00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1abc003f30 0x7f1ad00cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 -- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad0156150 msgr2=0x7f1ad01f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad0156150 0x7f1ad01f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.420+0000 7f1ad4aea640 1 -- 172.21.15.111:0/3772329508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad00cb8a0 con 0x7f1ad0156b80 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.421+0000 7f1ad4aea640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 0x7f1ad00cdbd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c300 tx=0x7f1ac4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.421+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4042070 con 0x7f1ad0156b80 2024-05-27T00:39:32.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.421+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac4007480 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.421+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4007d50 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.421+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad00cbb40 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.421+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad00cc100 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.422+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad0068680 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.422+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ac4045070 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.423+0000 7f1ac1ffb640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1ab406be30 0x7f1ab406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.423+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ac40abbb0 con 0x7f1ad0156b80 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.423+0000 7f1ad52eb640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1ab406be30 0x7f1ab406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.423+0000 7f1ad52eb640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1ab406be30 0x7f1ab406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ca3b0 tx=0x7f1acc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:32.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.427+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ac404c030 con 0x7f1ad0156b80 2024-05-27T00:39:32.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.601+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ad00cef10 con 0x7f1ad0156b80 2024-05-27T00:39:32.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.602+0000 7f1ac1ffb640 1 -- 172.21.15.111:0/3772329508 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f1ac4005ac0 con 0x7f1ad0156b80 2024-05-27T00:39:32.603 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:32.603 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:32.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1ab406be30 msgr2=0x7f1ab406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1ab406be30 0x7f1ab406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ca3b0 tx=0x7f1acc008040 comp rx=0 tx=0).stop 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 msgr2=0x7f1ad00cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 0x7f1ad00cdbd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c300 tx=0x7f1ac4000fe0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 shutdown_connections 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1ab406be30 0x7f1ab406e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ad0156150 0x7f1ad01f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0156b80 0x7f1ad00cdbd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 --2- 172.21.15.111:0/3772329508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1abc003f30 0x7f1ad00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 >> 172.21.15.111:0/3772329508 conn(0x7f1ad00d42c0 msgr2=0x7f1ad006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:32.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 shutdown_connections 2024-05-27T00:39:32.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.604+0000 7f1ad754f640 1 -- 172.21.15.111:0/3772329508 wait complete. 2024-05-27T00:39:32.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:39:32.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:32.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.718+0000 7fd20165d640 1 Processor -- start 2024-05-27T00:39:32.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.719+0000 7fd20165d640 1 -- start start 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.719+0000 7fd20165d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.719+0000 7fd20165d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 0x7fd1fc0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.719+0000 7fd20165d640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc0b3a20 con 0x7fd1fc162050 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.719+0000 7fd20165d640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc0b4630 con 0x7fd1fc162430 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.719+0000 7fd20165d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b3d00 con 0x7fd1fc0af970 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1f27fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1fb7fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 0x7fd1fc0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1f27fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc15f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:50762/0 (socket says 172.21.15.111:50762) 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1faffd640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd1fc162050 0x7fd1fc0af260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:52912/0 (socket says 172.21.15.111:52912) 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1fb7fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 0x7fd1fc0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:42062/0 (socket says 172.21.15.111:42062) 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1faffd640 1 -- 172.21.15.111:0/1098546783 learned_addr learned my addr 172.21.15.111:0/1098546783 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1f27fc640 1 -- 172.21.15.111:0/1098546783 >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 msgr2=0x7fd1fc0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1f27fc640 1 --2- 172.21.15.111:0/1098546783 >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 0x7fd1fc0b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1f27fc640 1 -- 172.21.15.111:0/1098546783 >> v1:172.21.15.153:6789/0 conn(0x7fd1fc162050 legacy=0x7fd1fc0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1f27fc640 1 -- 172.21.15.111:0/1098546783 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0b3fe0 con 0x7fd1fc0af970 2024-05-27T00:39:32.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.720+0000 7fd1fb7fe640 1 --2- 172.21.15.111:0/1098546783 >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 0x7fd1fc0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.721+0000 7fd1f27fc640 1 --2- 172.21.15.111:0/1098546783 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc15f9b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40097a0 tx=0x7fd1e402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35053fa5034ef91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.721+0000 7fd1fa7fc640 1 -- 172.21.15.111:0/1098546783 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4037070 con 0x7fd1fc0af970 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.721+0000 7fd1fa7fc640 1 -- 172.21.15.111:0/1098546783 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e4032440 con 0x7fd1fc0af970 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.721+0000 7fd1fa7fc640 1 -- 172.21.15.111:0/1098546783 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4032810 con 0x7fd1fc0af970 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 -- 172.21.15.111:0/1098546783 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 msgr2=0x7fd1fc15f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 --2- 172.21.15.111:0/1098546783 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc15f9b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40097a0 tx=0x7fd1e402a180 comp rx=0 tx=0).stop 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 -- 172.21.15.111:0/1098546783 shutdown_connections 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 --2- 172.21.15.111:0/1098546783 >> v2:172.21.15.153:3300/0 conn(0x7fd1fc162430 0x7fd1fc0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 --2- 172.21.15.111:0/1098546783 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc15f9b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 -- 172.21.15.111:0/1098546783 >> 172.21.15.111:0/1098546783 conn(0x7fd1fc072a30 msgr2=0x7fd1fc06bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:32.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 -- 172.21.15.111:0/1098546783 shutdown_connections 2024-05-27T00:39:32.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.722+0000 7fd20165d640 1 -- 172.21.15.111:0/1098546783 wait complete. 2024-05-27T00:39:32.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.723+0000 7fd20165d640 1 Processor -- start 2024-05-27T00:39:32.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.723+0000 7fd20165d640 1 -- start start 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc14b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 0x7fd1fc1e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1fc162430 0x7fd1fc1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b5180 con 0x7fd1fc162050 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b5000 con 0x7fd1fc162430 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b5300 con 0x7fd1fc0af970 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 0x7fd1fc1e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 0x7fd1fc1e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56566/0 (socket says 172.21.15.111:56566) 2024-05-27T00:39:32.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 -- 172.21.15.111:0/2393551677 learned_addr learned my addr 172.21.15.111:0/2393551677 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1faffd640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc14b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1fb7fe640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1fc162430 0x7fd1fc1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 -- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 msgr2=0x7fd1fc14b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc14b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 -- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1fc162430 msgr2=0x7fd1fc1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1fc162430 0x7fd1fc1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 -- 172.21.15.111:0/2393551677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc1e8660 con 0x7fd1fc162050 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f27fc640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 0x7fd1fc1e9910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40098c0 tx=0x7fd1e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4037040 con 0x7fd1fc162050 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e4030500 con 0x7fd1fc162050 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc1e8b90 con 0x7fd1fc162050 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.724+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40308d0 con 0x7fd1fc162050 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.725+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc1e9110 con 0x7fd1fc162050 2024-05-27T00:39:32.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.726+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc068680 con 0x7fd1fc162050 2024-05-27T00:39:32.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.726+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1e4044020 con 0x7fd1fc162050 2024-05-27T00:39:32.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.727+0000 7fd1f3fff640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1dc06be30 0x7fd1dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:32.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.727+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd1e40ab230 con 0x7fd1fc162050 2024-05-27T00:39:32.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.727+0000 7fd1faffd640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1dc06be30 0x7fd1dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:32.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.727+0000 7fd1faffd640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1dc06be30 0x7fd1dc06e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec007fd0 tx=0x7fd1ec007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:32.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.731+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd1e40792e0 con 0x7fd1fc162050 2024-05-27T00:39:32.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.904+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd1fc0b0e30 con 0x7fd1fc162050 2024-05-27T00:39:32.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.905+0000 7fd1f3fff640 1 -- 172.21.15.111:0/2393551677 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd1e407d5e0 con 0x7fd1fc162050 2024-05-27T00:39:32.906 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1dc06be30 msgr2=0x7fd1dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1dc06be30 0x7fd1dc06e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec007fd0 tx=0x7fd1ec007e80 comp rx=0 tx=0).stop 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 msgr2=0x7fd1fc1e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 0x7fd1fc1e9910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40098c0 tx=0x7fd1e40046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 shutdown_connections 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1dc06be30 0x7fd1dc06e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1fc162430 0x7fd1fc1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc162050 0x7fd1fc1e9910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 --2- 172.21.15.111:0/2393551677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1fc0af970 0x7fd1fc14b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 >> 172.21.15.111:0/2393551677 conn(0x7fd1fc072a30 msgr2=0x7fd1fc06ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:32.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 shutdown_connections 2024-05-27T00:39:32.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:32.907+0000 7fd20165d640 1 -- 172.21.15.111:0/2393551677 wait complete. 2024-05-27T00:39:32.919 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:39:33.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.020+0000 7f559e329640 1 Processor -- start 2024-05-27T00:39:33.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.020+0000 7f559e329640 1 -- start start 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559e329640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f559815c610 0x7f559815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559e329640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f559806fd90 0x7f559815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559e329640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559815d470 con 0x7f55980b3ed0 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559e329640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f55980ce240 con 0x7f559815c610 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559e329640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f55980b3a40 con 0x7f559806fd90 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f5597fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f559815c610 0x7f559815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559c8c6640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f55980b3ed0 0x7f55980b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35350/0 (socket says 172.21.15.111:35350) 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f559c8c6640 1 -- 172.21.15.111:0/2147066772 learned_addr learned my addr 172.21.15.111:0/2147066772 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f55977fe640 1 --2- 172.21.15.111:0/2147066772 >> v2:172.21.15.153:3300/0 conn(0x7f559806fd90 0x7f559815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f5597fff640 1 -- 172.21.15.111:0/2147066772 >> v2:172.21.15.153:3300/0 conn(0x7f559806fd90 msgr2=0x7f559815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:33.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f5597fff640 1 --2- 172.21.15.111:0/2147066772 >> v2:172.21.15.153:3300/0 conn(0x7f559806fd90 0x7f559815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f5597fff640 1 -- 172.21.15.111:0/2147066772 >> v1:172.21.15.111:6789/0 conn(0x7f55980b3ed0 legacy=0x7f55980b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f5596ffd640 1 -- 172.21.15.111:0/2147066772 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103395392 0 0) 0x7f5590001490 con 0x7f55980b3ed0 2024-05-27T00:39:33.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.021+0000 7f5597fff640 1 -- 172.21.15.111:0/2147066772 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55980ce470 con 0x7f559815c610 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f5597fff640 1 --2- 172.21.15.111:0/2147066772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f559815c9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55980cfcd0 tx=0x7f558802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba82eaa467eccd8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f5596ffd640 1 -- 172.21.15.111:0/2147066772 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588037070 con 0x7f559815c610 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f5596ffd640 1 -- 172.21.15.111:0/2147066772 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5588032440 con 0x7f559815c610 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f5596ffd640 1 -- 172.21.15.111:0/2147066772 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558802ae60 con 0x7f559815c610 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f559e329640 1 -- 172.21.15.111:0/2147066772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 msgr2=0x7f559815c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f559e329640 1 --2- 172.21.15.111:0/2147066772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f559815c9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55980cfcd0 tx=0x7f558802a110 comp rx=0 tx=0).stop 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f559e329640 1 -- 172.21.15.111:0/2147066772 shutdown_connections 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f559e329640 1 --2- 172.21.15.111:0/2147066772 >> v2:172.21.15.153:3300/0 conn(0x7f559806fd90 0x7f559815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f559e329640 1 --2- 172.21.15.111:0/2147066772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f559815c9f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.022+0000 7f559e329640 1 -- 172.21.15.111:0/2147066772 >> 172.21.15.111:0/2147066772 conn(0x7f5598072a30 msgr2=0x7f559806bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.023+0000 7f559e329640 1 -- 172.21.15.111:0/2147066772 shutdown_connections 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.023+0000 7f559e329640 1 -- 172.21.15.111:0/2147066772 wait complete. 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.024+0000 7f559e329640 1 Processor -- start 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.024+0000 7f559e329640 1 -- start start 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f559806fd90 0x7f559815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55980b3ed0 0x7f559806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f55980c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f55980ced10 con 0x7f559815c610 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f55980cea30 con 0x7f559806fd90 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f559815c300 con 0x7f55980b3ed0 2024-05-27T00:39:33.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f55980c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f55980c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56578/0 (socket says 172.21.15.111:56578) 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 -- 172.21.15.111:0/2575181824 learned_addr learned my addr 172.21.15.111:0/2575181824 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f55977fe640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55980b3ed0 0x7f559806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f5597fff640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f559806fd90 0x7f559815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 -- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55980b3ed0 msgr2=0x7f559806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55980b3ed0 0x7f559806f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 -- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f559806fd90 msgr2=0x7f559815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f559806fd90 0x7f559815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 -- 172.21.15.111:0/2575181824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559806f0e0 con 0x7f559815c610 2024-05-27T00:39:33.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f5597fff640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f559806fd90 0x7f559815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559c8c6640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f55980c7140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f559000c700 tx=0x7f559000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590018070 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559000ce10 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55980c6410 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559000a860 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.025+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55980c69d0 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.026+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55980c6200 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.027+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f559001b070 con 0x7f559815c610 2024-05-27T00:39:33.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.027+0000 7f5594ff9640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f558406be30 0x7f558406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.027+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f559008a790 con 0x7f559815c610 2024-05-27T00:39:33.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.027+0000 7f5597fff640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f558406be30 0x7f558406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.028+0000 7f5597fff640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f558406be30 0x7f558406e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5588034000 tx=0x7f558802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.032+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f559815d470 con 0x7f559815c610 2024-05-27T00:39:33.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.204+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5598068680 con 0x7f559815c610 2024-05-27T00:39:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.205+0000 7f5594ff9640 1 -- 172.21.15.111:0/2575181824 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f5590058e00 con 0x7f559815c610 2024-05-27T00:39:33.206 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:33.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.206+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f558406be30 msgr2=0x7f558406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.206+0000 7f559e329640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f558406be30 0x7f558406e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5588034000 tx=0x7f558802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:33.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.206+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 msgr2=0x7f55980c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.206+0000 7f559e329640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f55980c7140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f559000c700 tx=0x7f559000cbc0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 shutdown_connections 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f558406be30 0x7f558406e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f559815c610 0x7f55980c7140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f55980b3ed0 0x7f559806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 --2- 172.21.15.111:0/2575181824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f559806fd90 0x7f559815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 >> 172.21.15.111:0/2575181824 conn(0x7f5598072a30 msgr2=0x7f559806b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 shutdown_connections 2024-05-27T00:39:33.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.207+0000 7f559e329640 1 -- 172.21.15.111:0/2575181824 wait complete. 2024-05-27T00:39:33.219 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:39:33.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.320+0000 7fe8451b7640 1 Processor -- start 2024-05-27T00:39:33.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.320+0000 7fe8451b7640 1 -- start start 2024-05-27T00:39:33.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe8451b7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe8400d1810 0x7fe8400ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe8451b7640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe8400ceeb0 0x7fe8400ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe8451b7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8400d1c70 con 0x7fe840145be0 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe8451b7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe8400d0310 con 0x7fe8400d1810 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe8451b7640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe8400cc640 con 0x7fe8400ceeb0 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe8400d1810 0x7fe8400ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83ed9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe840145be0 0x7fe8400cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35364/0 (socket says 172.21.15.111:35364) 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83e59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe8400d1810 0x7fe8400ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56590/0 (socket says 172.21.15.111:56590) 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83ed9d640 1 -- 172.21.15.111:0/2514533897 learned_addr learned my addr 172.21.15.111:0/2514533897 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:33.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83f59e640 1 --2- 172.21.15.111:0/2514533897 >> v2:172.21.15.153:3300/0 conn(0x7fe8400ceeb0 0x7fe8400ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83e59c640 1 -- 172.21.15.111:0/2514533897 >> v2:172.21.15.153:3300/0 conn(0x7fe8400ceeb0 msgr2=0x7fe8400ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83e59c640 1 --2- 172.21.15.111:0/2514533897 >> v2:172.21.15.153:3300/0 conn(0x7fe8400ceeb0 0x7fe8400ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83e59c640 1 -- 172.21.15.111:0/2514533897 >> v1:172.21.15.111:6789/0 conn(0x7fe840145be0 legacy=0x7fe8400cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83e59c640 1 -- 172.21.15.111:0/2514533897 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8400c7960 con 0x7fe8400d1810 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.321+0000 7fe83dd9b640 1 -- 172.21.15.111:0/2514533897 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709696804 0 0) 0x7fe8400d1c70 con 0x7fe840145be0 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.322+0000 7fe83e59c640 1 --2- 172.21.15.111:0/2514533897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe8400ce970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8400c8b30 tx=0x7fe83402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439875ba8e538ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.322+0000 7fe83dd9b640 1 -- 172.21.15.111:0/2514533897 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834037070 con 0x7fe8400d1810 2024-05-27T00:39:33.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.322+0000 7fe83dd9b640 1 -- 172.21.15.111:0/2514533897 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe834032440 con 0x7fe8400d1810 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.322+0000 7fe83dd9b640 1 -- 172.21.15.111:0/2514533897 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83402ae60 con 0x7fe8400d1810 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.322+0000 7fe8451b7640 1 -- 172.21.15.111:0/2514533897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 msgr2=0x7fe8400ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.322+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2514533897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe8400ce970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8400c8b30 tx=0x7fe83402a110 comp rx=0 tx=0).stop 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.323+0000 7fe8451b7640 1 -- 172.21.15.111:0/2514533897 shutdown_connections 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.323+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2514533897 >> v2:172.21.15.153:3300/0 conn(0x7fe8400ceeb0 0x7fe8400ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.323+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2514533897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe8400ce970 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.323+0000 7fe8451b7640 1 -- 172.21.15.111:0/2514533897 >> 172.21.15.111:0/2514533897 conn(0x7fe8400d1f60 msgr2=0x7fe8400d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.323+0000 7fe8451b7640 1 -- 172.21.15.111:0/2514533897 shutdown_connections 2024-05-27T00:39:33.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.323+0000 7fe8451b7640 1 -- 172.21.15.111:0/2514533897 wait complete. 2024-05-27T00:39:33.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.324+0000 7fe8451b7640 1 Processor -- start 2024-05-27T00:39:33.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.324+0000 7fe8451b7640 1 -- start start 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8451b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8400ceeb0 0x7fe840139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8451b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe840137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8451b7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe840145be0 0x7fe8400b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8451b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe8400cd010 con 0x7fe8400d1810 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8451b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe8400c7f20 con 0x7fe8400ceeb0 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8451b7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe8400cd190 con 0x7fe840145be0 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe840137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe840137320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56602/0 (socket says 172.21.15.111:56602) 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 -- 172.21.15.111:0/2365150342 learned_addr learned my addr 172.21.15.111:0/2365150342 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83ed9d640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8400ceeb0 0x7fe840139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83f59e640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe840145be0 0x7fe8400b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 -- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe840145be0 msgr2=0x7fe8400b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe840145be0 0x7fe8400b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 -- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8400ceeb0 msgr2=0x7fe840139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8400ceeb0 0x7fe840139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 -- 172.21.15.111:0/2365150342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8400b60c0 con 0x7fe8400d1810 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83f59e640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe840145be0 0x7fe8400b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe83e59c640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe840137320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe834009890 tx=0x7fe8340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.325+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834037040 con 0x7fe8400d1810 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.326+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe834042070 con 0x7fe8400d1810 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.326+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834030960 con 0x7fe8400d1810 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.326+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8400b63a0 con 0x7fe8400d1810 2024-05-27T00:39:33.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.326+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8400b6960 con 0x7fe8400d1810 2024-05-27T00:39:33.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.327+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe840068680 con 0x7fe8400d1810 2024-05-27T00:39:33.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.327+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe834030b00 con 0x7fe8400d1810 2024-05-27T00:39:33.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.328+0000 7fe8277fe640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe82c06be30 0x7fe82c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.328+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8340ab970 con 0x7fe8400d1810 2024-05-27T00:39:33.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.328+0000 7fe83ed9d640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe82c06be30 0x7fe82c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.328+0000 7fe83ed9d640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe82c06be30 0x7fe82c06e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe830007fd0 tx=0x7fe830007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.332+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe83407a070 con 0x7fe8400d1810 2024-05-27T00:39:33.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.505+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8400b3120 con 0x7fe8400d1810 2024-05-27T00:39:33.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.506+0000 7fe8277fe640 1 -- 172.21.15.111:0/2365150342 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fe834079970 con 0x7fe8400d1810 2024-05-27T00:39:33.507 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe82c06be30 msgr2=0x7fe82c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe82c06be30 0x7fe82c06e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe830007fd0 tx=0x7fe830007de0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 msgr2=0x7fe840137320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe840137320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe834009890 tx=0x7fe8340046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 shutdown_connections 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe82c06be30 0x7fe82c06e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe840145be0 0x7fe8400b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8400d1810 0x7fe840137320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 --2- 172.21.15.111:0/2365150342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8400ceeb0 0x7fe840139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 >> 172.21.15.111:0/2365150342 conn(0x7fe8400d1f60 msgr2=0x7fe8400b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 shutdown_connections 2024-05-27T00:39:33.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.508+0000 7fe8451b7640 1 -- 172.21.15.111:0/2365150342 wait complete. 2024-05-27T00:39:33.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:39:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.622+0000 7f7c3c6c4640 1 Processor -- start 2024-05-27T00:39:33.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.622+0000 7f7c3c6c4640 1 -- start start 2024-05-27T00:39:33.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3c6c4640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c340cf020 0x7f7c340d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3c6c4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340ce0e0 0x7f7c3415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3c6c4640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c340b4e40 con 0x7f7c3415c0b0 2024-05-27T00:39:33.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3c6c4640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f7c340ccbf0 con 0x7f7c340cf020 2024-05-27T00:39:33.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3c6c4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7c34070fe0 con 0x7f7c340ce0e0 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c39c5f640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c340cf020 0x7f7c340d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3ac61640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340ce0e0 0x7f7c3415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c39c5f640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c340cf020 0x7f7c340d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:42102/0 (socket says 172.21.15.111:42102) 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c39c5f640 1 -- 172.21.15.111:0/1358512047 learned_addr learned my addr 172.21.15.111:0/1358512047 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3ac61640 1 -- 172.21.15.111:0/1358512047 >> v2:172.21.15.153:3300/0 conn(0x7f7c340cf020 msgr2=0x7f7c340d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3ac61640 1 --2- 172.21.15.111:0/1358512047 >> v2:172.21.15.153:3300/0 conn(0x7f7c340cf020 0x7f7c340d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3ac61640 1 -- 172.21.15.111:0/1358512047 >> v1:172.21.15.153:6789/0 conn(0x7f7c3415c0b0 legacy=0x7f7c340ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.623+0000 7f7c3ac61640 1 -- 172.21.15.111:0/1358512047 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c34070a40 con 0x7f7c340ce0e0 2024-05-27T00:39:33.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.624+0000 7f7c3ac61640 1 --2- 172.21.15.111:0/1358512047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340ce0e0 0x7f7c3415fcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009d90 tx=0x7f7c2c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b741b6399c27ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.624+0000 7f7c3945e640 1 -- 172.21.15.111:0/1358512047 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c031600 con 0x7f7c340ce0e0 2024-05-27T00:39:33.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3945e640 1 -- 172.21.15.111:0/1358512047 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c0317a0 con 0x7f7c340ce0e0 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3945e640 1 -- 172.21.15.111:0/1358512047 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c031b70 con 0x7f7c340ce0e0 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/1358512047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340ce0e0 msgr2=0x7f7c3415fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/1358512047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340ce0e0 0x7f7c3415fcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009d90 tx=0x7f7c2c02a5d0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/1358512047 shutdown_connections 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/1358512047 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340ce0e0 0x7f7c3415fcc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/1358512047 >> v2:172.21.15.153:3300/0 conn(0x7f7c340cf020 0x7f7c340d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/1358512047 >> 172.21.15.111:0/1358512047 conn(0x7f7c3406ba70 msgr2=0x7f7c340d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/1358512047 shutdown_connections 2024-05-27T00:39:33.627 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.625+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/1358512047 wait complete. 2024-05-27T00:39:33.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.627+0000 7f7c3c6c4640 1 Processor -- start 2024-05-27T00:39:33.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.627+0000 7f7c3c6c4640 1 -- start start 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3c6c4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 0x7f7c3414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3c6c4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340cf020 0x7f7c34154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3c6c4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c3415c0b0 0x7f7c34154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3c6c4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340cc6d0 con 0x7f7c340ce0e0 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3c6c4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34070d00 con 0x7f7c3415c0b0 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3c6c4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3415c490 con 0x7f7c340cf020 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 0x7f7c3414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 0x7f7c3414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56610/0 (socket says 172.21.15.111:56610) 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 -- 172.21.15.111:0/3533017098 learned_addr learned my addr 172.21.15.111:0/3533017098 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:33.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c39c5f640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340cf020 0x7f7c34154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3ac61640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c3415c0b0 0x7f7c34154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 -- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340cf020 msgr2=0x7f7c34154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340cf020 0x7f7c34154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 -- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c3415c0b0 msgr2=0x7f7c34154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c3415c0b0 0x7f7c34154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 -- 172.21.15.111:0/3533017098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3414f4b0 con 0x7f7c340ce0e0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c3a460640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 0x7f7c3414ef70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c400 tx=0x7f7c3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.628+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300052d0 con 0x7f7c340ce0e0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.629+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c30005490 con 0x7f7c340ce0e0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.629+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30007260 con 0x7f7c340ce0e0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.629+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c34152ba0 con 0x7f7c340ce0e0 2024-05-27T00:39:33.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.629+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c34153160 con 0x7f7c340ce0e0 2024-05-27T00:39:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.630+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c34068680 con 0x7f7c340ce0e0 2024-05-27T00:39:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.630+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c30007400 con 0x7f7c340ce0e0 2024-05-27T00:39:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.630+0000 7f7c2affd640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7c1806be30 0x7f7c1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.631+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7c3008a750 con 0x7f7c340ce0e0 2024-05-27T00:39:33.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.631+0000 7f7c39c5f640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7c1806be30 0x7f7c1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.631+0000 7f7c39c5f640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7c1806be30 0x7f7c1806e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c340c91b0 tx=0x7f7c24009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.635+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7c30058660 con 0x7f7c340ce0e0 2024-05-27T00:39:33.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.806+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7c3415fa30 con 0x7f7c340ce0e0 2024-05-27T00:39:33.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.807+0000 7f7c2affd640 1 -- 172.21.15.111:0/3533017098 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f7c3005c960 con 0x7f7c340ce0e0 2024-05-27T00:39:33.808 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7c1806be30 msgr2=0x7f7c1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7c1806be30 0x7f7c1806e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c340c91b0 tx=0x7f7c24009240 comp rx=0 tx=0).stop 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 msgr2=0x7f7c3414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 0x7f7c3414ef70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c400 tx=0x7f7c3000c8c0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 shutdown_connections 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7c1806be30 0x7f7c1806e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c3415c0b0 0x7f7c34154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7c340cf020 0x7f7c34154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 --2- 172.21.15.111:0/3533017098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c340ce0e0 0x7f7c3414ef70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.808+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 >> 172.21.15.111:0/3533017098 conn(0x7f7c3406ba70 msgr2=0x7f7c340b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.809+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 shutdown_connections 2024-05-27T00:39:33.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.809+0000 7f7c3c6c4640 1 -- 172.21.15.111:0/3533017098 wait complete. 2024-05-27T00:39:33.820 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T00:39:33.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.920+0000 7f681f4b3640 1 Processor -- start 2024-05-27T00:39:33.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.920+0000 7f681f4b3640 1 -- start start 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681f4b3640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f681815e830 0x7f68181459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681f4b3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6818070030 0x7f68180704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681f4b3640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6818145f90 con 0x7f6818144a10 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681f4b3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f68180709e0 con 0x7f6818070030 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681f4b3640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6818146270 con 0x7f681815e830 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6818070030 0x7f68180704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6818070030 0x7f68180704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56624/0 (socket says 172.21.15.111:56624) 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 -- 172.21.15.111:0/3626166463 learned_addr learned my addr 172.21.15.111:0/3626166463 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:33.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681ca4e640 1 --2- 172.21.15.111:0/3626166463 >> v2:172.21.15.153:3300/0 conn(0x7f681815e830 0x7f68181459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 -- 172.21.15.111:0/3626166463 >> v2:172.21.15.153:3300/0 conn(0x7f681815e830 msgr2=0x7f68181459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 --2- 172.21.15.111:0/3626166463 >> v2:172.21.15.153:3300/0 conn(0x7f681815e830 0x7f68181459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 -- 172.21.15.111:0/3626166463 >> v1:172.21.15.153:6789/0 conn(0x7f6818144a10 legacy=0x7f68180a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.921+0000 7f681da50640 1 -- 172.21.15.111:0/3626166463 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6818146550 con 0x7f6818070030 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.922+0000 7f681da50640 1 --2- 172.21.15.111:0/3626166463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818070030 0x7f68180704a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6810009e50 tx=0x7f681002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20846abd28e73cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.922+0000 7f680ffff640 1 -- 172.21.15.111:0/3626166463 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810036070 con 0x7f6818070030 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.922+0000 7f680ffff640 1 -- 172.21.15.111:0/3626166463 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6810031600 con 0x7f6818070030 2024-05-27T00:39:33.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.922+0000 7f680ffff640 1 -- 172.21.15.111:0/3626166463 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100319d0 con 0x7f6818070030 2024-05-27T00:39:33.926 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 -- 172.21.15.111:0/3626166463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818070030 msgr2=0x7f68180704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 --2- 172.21.15.111:0/3626166463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818070030 0x7f68180704a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6810009e50 tx=0x7f681002a760 comp rx=0 tx=0).stop 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 -- 172.21.15.111:0/3626166463 shutdown_connections 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 --2- 172.21.15.111:0/3626166463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818070030 0x7f68180704a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 --2- 172.21.15.111:0/3626166463 >> v2:172.21.15.153:3300/0 conn(0x7f681815e830 0x7f68181459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 -- 172.21.15.111:0/3626166463 >> 172.21.15.111:0/3626166463 conn(0x7f6818073130 msgr2=0x7f68180b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 -- 172.21.15.111:0/3626166463 shutdown_connections 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.923+0000 7f681f4b3640 1 -- 172.21.15.111:0/3626166463 wait complete. 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.924+0000 7f681f4b3640 1 Processor -- start 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.924+0000 7f681f4b3640 1 -- start start 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6818070030 0x7f6818150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 0x7f681813d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f681815e830 0x7f681813dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6818147030 con 0x7f6818144a10 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6818146b10 con 0x7f6818070030 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f68181471b0 con 0x7f681815e830 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 0x7f681813d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 0x7f681813d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56636/0 (socket says 172.21.15.111:56636) 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 -- 172.21.15.111:0/684406858 learned_addr learned my addr 172.21.15.111:0/684406858 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681d24f640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6818070030 0x7f6818150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681da50640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f681815e830 0x7f681813dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 -- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f681815e830 msgr2=0x7f681813dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f681815e830 0x7f681813dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 -- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6818070030 msgr2=0x7f6818150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6818070030 0x7f6818150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 -- 172.21.15.111:0/684406858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6818151180 con 0x7f6818144a10 2024-05-27T00:39:33.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681ca4e640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 0x7f681813d6d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f680800e8a0 tx=0x7f680800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808017070 con 0x7f6818144a10 2024-05-27T00:39:33.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680800c940 con 0x7f6818144a10 2024-05-27T00:39:33.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6818152430 con 0x7f6818144a10 2024-05-27T00:39:33.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.925+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680800cd30 con 0x7f6818144a10 2024-05-27T00:39:33.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.926+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68181529f0 con 0x7f6818144a10 2024-05-27T00:39:33.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.927+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6818068680 con 0x7f6818144a10 2024-05-27T00:39:33.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.927+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f680801a070 con 0x7f6818144a10 2024-05-27T00:39:33.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.927+0000 7f680dffb640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f67fc06be60 0x7f67fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:33.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.927+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f680808b300 con 0x7f6818144a10 2024-05-27T00:39:33.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.928+0000 7f681d24f640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f67fc06be60 0x7f67fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:33.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.928+0000 7f681d24f640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f67fc06be60 0x7f67fc06e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6814001780 tx=0x7f6814009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:33.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:33.932+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6808059320 con 0x7f6818144a10 2024-05-27T00:39:34.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.120+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68180d04d0 con 0x7f6818144a10 2024-05-27T00:39:34.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.120+0000 7f680dffb640 1 -- 172.21.15.111:0/684406858 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f680805d620 con 0x7f6818144a10 2024-05-27T00:39:34.121 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T00:39:34.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f67fc06be60 msgr2=0x7f67fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f67fc06be60 0x7f67fc06e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6814001780 tx=0x7f6814009040 comp rx=0 tx=0).stop 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 msgr2=0x7f681813d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 0x7f681813d6d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f680800e8a0 tx=0x7f680800eca0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 shutdown_connections 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f67fc06be60 0x7f67fc06e320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f681815e830 0x7f681813dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6818144a10 0x7f681813d6d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 --2- 172.21.15.111:0/684406858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6818070030 0x7f6818150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 >> 172.21.15.111:0/684406858 conn(0x7f6818073130 msgr2=0x7f681815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 shutdown_connections 2024-05-27T00:39:34.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.122+0000 7f681f4b3640 1 -- 172.21.15.111:0/684406858 wait complete. 2024-05-27T00:39:34.134 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:39:34.134 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:39:34.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:34.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.288+0000 7f79226b3640 1 Processor -- start 2024-05-27T00:39:34.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.288+0000 7f79226b3640 1 -- start start 2024-05-27T00:39:34.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.289+0000 7f79226b3640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f791c144000 0x7f791c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.289+0000 7f79226b3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c0726d0 0x7f791c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.289+0000 7f79226b3640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f791c073020 con 0x7f791c06b390 2024-05-27T00:39:34.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.289+0000 7f79226b3640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f791c0b42f0 con 0x7f791c144000 2024-05-27T00:39:34.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.289+0000 7f79226b3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b3440 con 0x7f791c0726d0 2024-05-27T00:39:34.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f791c144000 0x7f791c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f7920c50640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c0726d0 0x7f791c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f791c144000 0x7f791c0b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:42140/0 (socket says 172.21.15.111:42140) 2024-05-27T00:39:34.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 -- 172.21.15.111:0/187148214 learned_addr learned my addr 172.21.15.111:0/187148214 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:34.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791affd640 1 -- 172.21.15.111:0/187148214 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897521164 0 0) 0x7f7910001110 con 0x7f791c06b390 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791affd640 1 -- 172.21.15.111:0/187148214 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7904003520 con 0x7f791c06b390 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 -- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c0726d0 msgr2=0x7f791c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 --2- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c0726d0 0x7f791c072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 -- 172.21.15.111:0/187148214 >> v1:172.21.15.153:6789/0 conn(0x7f791c06b390 legacy=0x7f791c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f791bfff640 1 -- 172.21.15.111:0/187148214 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0b36e0 con 0x7f791c144000 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.290+0000 7f7920c50640 1 --2- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c0726d0 0x7f791c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f791bfff640 1 --2- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c144000 0x7f791c0b2f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f790c009870 tx=0x7f790c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cadeb1c29969f58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f791affd640 1 -- 172.21.15.111:0/187148214 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c037070 con 0x7f791c144000 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f791affd640 1 -- 172.21.15.111:0/187148214 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c032440 con 0x7f791c144000 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f791affd640 1 -- 172.21.15.111:0/187148214 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c0327f0 con 0x7f791c144000 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 -- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c144000 msgr2=0x7f791c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 --2- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c144000 0x7f791c0b2f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f790c009870 tx=0x7f790c02a270 comp rx=0 tx=0).stop 2024-05-27T00:39:34.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 -- 172.21.15.111:0/187148214 shutdown_connections 2024-05-27T00:39:34.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 --2- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c0726d0 0x7f791c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 --2- 172.21.15.111:0/187148214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c144000 0x7f791c0b2f00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 -- 172.21.15.111:0/187148214 >> 172.21.15.111:0/187148214 conn(0x7f791c0b1c00 msgr2=0x7f791c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:34.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 -- 172.21.15.111:0/187148214 shutdown_connections 2024-05-27T00:39:34.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.291+0000 7f79226b3640 1 -- 172.21.15.111:0/187148214 wait complete. 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 Processor -- start 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 -- start start 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c06b390 0x7f791c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c0726d0 0x7f791c167660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 0x7f791c167ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b3f80 con 0x7f791c144000 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b3ca0 con 0x7f791c0726d0 2024-05-27T00:39:34.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.293+0000 7f79226b3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0cde50 con 0x7f791c06b390 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 0x7f791c167ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 0x7f791c167ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56646/0 (socket says 172.21.15.111:56646) 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 -- 172.21.15.111:0/3682560464 learned_addr learned my addr 172.21.15.111:0/3682560464 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f791bfff640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c06b390 0x7f791c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f791b7fe640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c0726d0 0x7f791c167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 -- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c06b390 msgr2=0x7f791c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c06b390 0x7f791c0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 -- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c0726d0 msgr2=0x7f791c167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c0726d0 0x7f791c167660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 -- 172.21.15.111:0/3682560464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0aebe0 con 0x7f791c144000 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f791b7fe640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c0726d0 0x7f791c167660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7920c50640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 0x7f791c167ba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f791400efc0 tx=0x7f7914009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914018030 con 0x7f791c144000 2024-05-27T00:39:34.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791400cde0 con 0x7f791c144000 2024-05-27T00:39:34.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c0aeec0 con 0x7f791c144000 2024-05-27T00:39:34.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914003bb0 con 0x7f791c144000 2024-05-27T00:39:34.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.294+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c139af0 con 0x7f791c144000 2024-05-27T00:39:34.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.295+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c139cf0 con 0x7f791c144000 2024-05-27T00:39:34.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.296+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f791401b070 con 0x7f791c144000 2024-05-27T00:39:34.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.296+0000 7f7918ff9640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f790806be30 0x7f790806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.296+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f791408bb80 con 0x7f791c144000 2024-05-27T00:39:34.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.296+0000 7f791bfff640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f790806be30 0x7f790806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.297+0000 7f791bfff640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f790806be30 0x7f790806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f790c034000 tx=0x7f790c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.301+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f791c139cf0 con 0x7f791c144000 2024-05-27T00:39:34.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.477+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f791c068680 con 0x7f791c144000 2024-05-27T00:39:34.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.478+0000 7f7918ff9640 1 -- 172.21.15.111:0/3682560464 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f7914010040 con 0x7f791c144000 2024-05-27T00:39:34.479 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:34.479 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f790806be30 msgr2=0x7f790806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f790806be30 0x7f790806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f790c034000 tx=0x7f790c02e000 comp rx=0 tx=0).stop 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 msgr2=0x7f791c167ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 0x7f791c167ba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f791400efc0 tx=0x7f7914009f90 comp rx=0 tx=0).stop 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 shutdown_connections 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f790806be30 0x7f790806e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f791c144000 0x7f791c167ba0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f791c0726d0 0x7f791c167660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 --2- 172.21.15.111:0/3682560464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f791c06b390 0x7f791c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 >> 172.21.15.111:0/3682560464 conn(0x7f791c0b1c00 msgr2=0x7f791c071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:34.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 shutdown_connections 2024-05-27T00:39:34.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.481+0000 7f79226b3640 1 -- 172.21.15.111:0/3682560464 wait complete. 2024-05-27T00:39:34.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:39:34.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:34.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.594+0000 7fe3b12ae640 1 Processor -- start 2024-05-27T00:39:34.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.594+0000 7fe3b12ae640 1 -- start start 2024-05-27T00:39:34.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3b12ae640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe3ac0b3630 0x7fe3ac0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3b12ae640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac162560 0x7fe3ac0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3b12ae640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3ac06cc70 con 0x7fe3ac0b77e0 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3b12ae640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe3ac0d1e50 con 0x7fe3ac0b3630 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3b12ae640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ac0d2130 con 0x7fe3ac162560 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe3ac0b3630 0x7fe3ac0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe3ac0b3630 0x7fe3ac0b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56660/0 (socket says 172.21.15.111:56660) 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3161658362 learned_addr learned my addr 172.21.15.111:0/3161658362 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3161658362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac162560 msgr2=0x7fe3ac0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3ab7fe640 1 --2- 172.21.15.111:0/3161658362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac162560 0x7fe3ac0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 --2- 172.21.15.111:0/3161658362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac162560 0x7fe3ac0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3161658362 >> v1:172.21.15.153:6789/0 conn(0x7fe3ac0b77e0 legacy=0x7fe3ac161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:34.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.595+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3161658362 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac0d2410 con 0x7fe3ac0b3630 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.596+0000 7fe3aaffd640 1 --2- 172.21.15.111:0/3161658362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac0b72a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009e50 tx=0x7fe39c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2392ed6af7f709b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.596+0000 7fe3a9ffb640 1 -- 172.21.15.111:0/3161658362 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c036070 con 0x7fe3ac0b3630 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.596+0000 7fe3a9ffb640 1 -- 172.21.15.111:0/3161658362 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39c031600 con 0x7fe3ac0b3630 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.596+0000 7fe3a9ffb640 1 -- 172.21.15.111:0/3161658362 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0319d0 con 0x7fe3ac0b3630 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3161658362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 msgr2=0x7fe3ac0b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3161658362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac0b72a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009e50 tx=0x7fe39c02a6c0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3161658362 shutdown_connections 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3161658362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac162560 0x7fe3ac0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3161658362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac0b72a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3161658362 >> 172.21.15.111:0/3161658362 conn(0x7fe3ac0d42c0 msgr2=0x7fe3ac0d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:34.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3161658362 shutdown_connections 2024-05-27T00:39:34.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.597+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3161658362 wait complete. 2024-05-27T00:39:34.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.598+0000 7fe3b12ae640 1 Processor -- start 2024-05-27T00:39:34.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.598+0000 7fe3b12ae640 1 -- start start 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3b12ae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3b12ae640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac0b77e0 0x7fe3ac0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3b12ae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ac162560 0x7fe3ac0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3b12ae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac0d32d0 con 0x7fe3ac0b3630 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3b12ae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac0d3150 con 0x7fe3ac162560 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3b12ae640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ac0d3450 con 0x7fe3ac0b77e0 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56674/0 (socket says 172.21.15.111:56674) 2024-05-27T00:39:34.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3770979282 learned_addr learned my addr 172.21.15.111:0/3770979282 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3ab7fe640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ac162560 0x7fe3ac0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aa7fc640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac0b77e0 0x7fe3ac0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac0b77e0 msgr2=0x7fe3ac0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac0b77e0 0x7fe3ac0c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ac162560 msgr2=0x7fe3ac0c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ac162560 0x7fe3ac0c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.599+0000 7fe3aaffd640 1 -- 172.21.15.111:0/3770979282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac1f5050 con 0x7fe3ac0b3630 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.600+0000 7fe3aaffd640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac15b780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009e50 tx=0x7fe39c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.600+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c036050 con 0x7fe3ac0b3630 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.600+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39c042070 con 0x7fe3ac0b3630 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.600+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c009510 con 0x7fe3ac0b3630 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.600+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac1e7af0 con 0x7fe3ac0b3630 2024-05-27T00:39:34.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.600+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac15bcc0 con 0x7fe3ac0b3630 2024-05-27T00:39:34.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.601+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ac068680 con 0x7fe3ac0b3630 2024-05-27T00:39:34.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.601+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe39c007c70 con 0x7fe3ac0b3630 2024-05-27T00:39:34.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.601+0000 7fe3937fe640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe39806be30 0x7fe39806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.602+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe39c0ac100 con 0x7fe3ac0b3630 2024-05-27T00:39:34.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.602+0000 7fe3aa7fc640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe39806be30 0x7fe39806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.602+0000 7fe3aa7fc640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe39806be30 0x7fe39806e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac15c110 tx=0x7fe3a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.606+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe39c045360 con 0x7fe3ac0b3630 2024-05-27T00:39:34.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.780+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3ac0cef10 con 0x7fe3ac0b3630 2024-05-27T00:39:34.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.780+0000 7fe3937fe640 1 -- 172.21.15.111:0/3770979282 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe39c07a800 con 0x7fe3ac0b3630 2024-05-27T00:39:34.782 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:34.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe39806be30 msgr2=0x7fe39806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe39806be30 0x7fe39806e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac15c110 tx=0x7fe3a0009040 comp rx=0 tx=0).stop 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 msgr2=0x7fe3ac15b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac15b780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009e50 tx=0x7fe39c004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 shutdown_connections 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe39806be30 0x7fe39806e2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3ac162560 0x7fe3ac0c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe3ac0b77e0 0x7fe3ac0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 --2- 172.21.15.111:0/3770979282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0b3630 0x7fe3ac15b780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.782+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 >> 172.21.15.111:0/3770979282 conn(0x7fe3ac0d42c0 msgr2=0x7fe3ac06b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.783+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 shutdown_connections 2024-05-27T00:39:34.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.783+0000 7fe3b12ae640 1 -- 172.21.15.111:0/3770979282 wait complete. 2024-05-27T00:39:34.795 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:39:34.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.896+0000 7f19f7261640 1 Processor -- start 2024-05-27T00:39:34.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.896+0000 7f19f7261640 1 -- start start 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f7261640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19f0162050 0x7f19f015f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f7261640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19f0162430 0x7f19f00af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f7261640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f00afc30 con 0x7f19f00af7a0 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f7261640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f19f00b4630 con 0x7f19f0162050 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f7261640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f19f00b3a20 con 0x7f19f0162430 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19f0162050 0x7f19f015f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f19f0162050 0x7f19f015f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56684/0 (socket says 172.21.15.111:56684) 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 -- 172.21.15.111:0/3883974169 learned_addr learned my addr 172.21.15.111:0/3883974169 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19e7fff640 1 --2- 172.21.15.111:0/3883974169 >> v2:172.21.15.153:3300/0 conn(0x7f19f0162430 0x7f19f00af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 -- 172.21.15.111:0/3883974169 >> v2:172.21.15.153:3300/0 conn(0x7f19f0162430 msgr2=0x7f19f00af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 --2- 172.21.15.111:0/3883974169 >> v2:172.21.15.153:3300/0 conn(0x7f19f0162430 0x7f19f00af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 -- 172.21.15.111:0/3883974169 >> v1:172.21.15.153:6789/0 conn(0x7f19f00af7a0 legacy=0x7f19f00b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.897+0000 7f19f4ffd640 1 -- 172.21.15.111:0/3883974169 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00b3d00 con 0x7f19f0162050 2024-05-27T00:39:34.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.898+0000 7f19f4ffd640 1 --2- 172.21.15.111:0/3883974169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162050 0x7f19f015f9b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19f0075060 tx=0x7f19ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea23768c53edaac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.898+0000 7f19e77fe640 1 -- 172.21.15.111:0/3883974169 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec037070 con 0x7f19f0162050 2024-05-27T00:39:34.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.898+0000 7f19e77fe640 1 -- 172.21.15.111:0/3883974169 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec032440 con 0x7f19f0162050 2024-05-27T00:39:34.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.898+0000 7f19e77fe640 1 -- 172.21.15.111:0/3883974169 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec032810 con 0x7f19f0162050 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.898+0000 7f19f7261640 1 -- 172.21.15.111:0/3883974169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162050 msgr2=0x7f19f015f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.898+0000 7f19f7261640 1 --2- 172.21.15.111:0/3883974169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162050 0x7f19f015f9b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19f0075060 tx=0x7f19ec02a270 comp rx=0 tx=0).stop 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.899+0000 7f19f7261640 1 -- 172.21.15.111:0/3883974169 shutdown_connections 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.899+0000 7f19f7261640 1 --2- 172.21.15.111:0/3883974169 >> v2:172.21.15.153:3300/0 conn(0x7f19f0162430 0x7f19f00af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.899+0000 7f19f7261640 1 --2- 172.21.15.111:0/3883974169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162050 0x7f19f015f9b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.899+0000 7f19f7261640 1 -- 172.21.15.111:0/3883974169 >> 172.21.15.111:0/3883974169 conn(0x7f19f0072a30 msgr2=0x7f19f006bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.899+0000 7f19f7261640 1 -- 172.21.15.111:0/3883974169 shutdown_connections 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.899+0000 7f19f7261640 1 -- 172.21.15.111:0/3883974169 wait complete. 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.900+0000 7f19f7261640 1 Processor -- start 2024-05-27T00:39:34.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.900+0000 7f19f7261640 1 -- start start 2024-05-27T00:39:34.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19f00af7a0 0x7f19f014a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19f0162050 0x7f19f014adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 0x7f19f014b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00b51d0 con 0x7f19f0162430 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00b42c0 con 0x7f19f0162050 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00b5350 con 0x7f19f00af7a0 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 0x7f19f014b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 0x7f19f014b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56692/0 (socket says 172.21.15.111:56692) 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 -- 172.21.15.111:0/686311854 learned_addr learned my addr 172.21.15.111:0/686311854 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19e7fff640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19f0162050 0x7f19f014adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f4ffd640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19f00af7a0 0x7f19f014a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 -- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19f00af7a0 msgr2=0x7f19f014a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19f00af7a0 0x7f19f014a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 -- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19f0162050 msgr2=0x7f19f014adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19f0162050 0x7f19f014adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 -- 172.21.15.111:0/686311854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0153420 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f57fe640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 0x7f19f014b300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c7a0 tx=0x7f19e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8003b70 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8003d30 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01536c0 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800aa90 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.901+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f0153c80 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.902+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f0068680 con 0x7f19f0162430 2024-05-27T00:39:34.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.903+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19e800ac30 con 0x7f19f0162430 2024-05-27T00:39:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.903+0000 7f19e57fa640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f19dc06be30 0x7f19dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.903+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f19e808afd0 con 0x7f19f0162430 2024-05-27T00:39:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.903+0000 7f19f4ffd640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f19dc06be30 0x7f19dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:34.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.904+0000 7f19f4ffd640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f19dc06be30 0x7f19dc06e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19ec034000 tx=0x7f19ec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:34.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:34.907+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f19e80596d0 con 0x7f19f0162430 2024-05-27T00:39:35.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.080+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19f006ed40 con 0x7f19f0162430 2024-05-27T00:39:35.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.080+0000 7f19e57fa640 1 -- 172.21.15.111:0/686311854 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f19e8058fd0 con 0x7f19f0162430 2024-05-27T00:39:35.081 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:35.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f19dc06be30 msgr2=0x7f19dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f19dc06be30 0x7f19dc06e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19ec034000 tx=0x7f19ec02e000 comp rx=0 tx=0).stop 2024-05-27T00:39:35.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 msgr2=0x7f19f014b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 0x7f19f014b300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c7a0 tx=0x7f19e800cc60 comp rx=0 tx=0).stop 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 shutdown_connections 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f19dc06be30 0x7f19dc06e2f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f0162430 0x7f19f014b300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19f0162050 0x7f19f014adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 --2- 172.21.15.111:0/686311854 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f19f00af7a0 0x7f19f014a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 >> 172.21.15.111:0/686311854 conn(0x7f19f0072a30 msgr2=0x7f19f006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 shutdown_connections 2024-05-27T00:39:35.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.082+0000 7f19f7261640 1 -- 172.21.15.111:0/686311854 wait complete. 2024-05-27T00:39:35.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:39:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.195+0000 7fe2b336a640 1 Processor -- start 2024-05-27T00:39:35.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.195+0000 7fe2b336a640 1 -- start start 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.196+0000 7fe2b336a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 0x7fe2ac0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.196+0000 7fe2b336a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac142650 con 0x7fe2ac0cce00 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.196+0000 7fe2b336a640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac0cebc0 con 0x7fe2ac13ef60 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.196+0000 7fe2b336a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cf210 con 0x7fe2ac13f340 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.196+0000 7fe2b1907640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe2ac0cce00 0x7fe2ac141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35372/0 (socket says 172.21.15.111:35372) 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.196+0000 7fe2b1907640 1 -- 172.21.15.111:0/717349817 learned_addr learned my addr 172.21.15.111:0/717349817 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b0905640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 0x7fe2ac0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747410318 0 0) 0x7fe2ac142650 con 0x7fe2ac0cce00 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe298003520 con 0x7fe2ac0cce00 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2029955727 0 0) 0x7fe298003520 con 0x7fe2ac0cce00 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 msgr2=0x7fe2ac0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 0x7fe2ac0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 >> v1:172.21.15.153:6789/0 conn(0x7fe2ac13ef60 legacy=0x7fe2ac0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac0ceea0 con 0x7fe2ac0cce00 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fe2a4001670 con 0x7fe2ac0cce00 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 >> v1:172.21.15.111:6789/0 conn(0x7fe2ac0cce00 legacy=0x7fe2ac141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe298003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe298004390 0x7fe298004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 0x7fe2980050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cf210 con 0x7fe2980039e0 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe29800b9c0 con 0x7fe298004390 2024-05-27T00:39:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe29800bbc0 con 0x7fe2ac13f340 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1106640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe298003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1907640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 0x7fe2980050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b0905640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe298004390 0x7fe298004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1106640 1 -- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 msgr2=0x7fe2980050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1106640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac13f340 0x7fe2980050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1106640 1 -- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe298004390 msgr2=0x7fe298004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1106640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe298004390 0x7fe298004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b1106640 1 -- 172.21.15.111:0/717349817 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298003520 con 0x7fe2980039e0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.197+0000 7fe2b0905640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe298004390 0x7fe298004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.198+0000 7fe2b1106640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe298003dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8001f60 tx=0x7fe298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf2cdd7dd85ca41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.198+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8001b30 con 0x7fe2980039e0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.198+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a8031620 con 0x7fe2980039e0 2024-05-27T00:39:35.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.198+0000 7fe2a3fff640 1 -- 172.21.15.111:0/717349817 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8031a10 con 0x7fe2980039e0 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b336a640 1 -- 172.21.15.111:0/717349817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 msgr2=0x7fe298003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b336a640 1 --2- 172.21.15.111:0/717349817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe298003dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8001f60 tx=0x7fe298005830 comp rx=0 tx=0).stop 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b1907640 1 -- 172.21.15.111:0/717349817 reap_dead start 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b336a640 1 -- 172.21.15.111:0/717349817 shutdown_connections 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b336a640 1 -- 172.21.15.111:0/717349817 >> 172.21.15.111:0/717349817 conn(0x7fe2ac0b1c00 msgr2=0x7fe2ac0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b336a640 1 -- 172.21.15.111:0/717349817 shutdown_connections 2024-05-27T00:39:35.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.199+0000 7fe2b336a640 1 -- 172.21.15.111:0/717349817 wait complete. 2024-05-27T00:39:35.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.200+0000 7fe2b336a640 1 Processor -- start 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.200+0000 7fe2b336a640 1 -- start start 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b336a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe2ac15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b336a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac132290 0x7fe2ac155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b336a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ac144d20 0x7fe2ac1451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b336a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cca40 con 0x7fe2980039e0 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b336a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cc8c0 con 0x7fe2ac144d20 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b336a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0ccbc0 con 0x7fe2ac132290 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe2ac15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe2ac15a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56714/0 (socket says 172.21.15.111:56714) 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 -- 172.21.15.111:0/144313460 learned_addr learned my addr 172.21.15.111:0/144313460 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 -- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac132290 msgr2=0x7fe2ac155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1907640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ac144d20 0x7fe2ac1451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b0905640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac132290 0x7fe2ac155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac132290 0x7fe2ac155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 -- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ac144d20 msgr2=0x7fe2ac1451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ac144d20 0x7fe2ac1451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 -- 172.21.15.111:0/144313460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac13f7c0 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2b1106640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe2ac15a200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8002080 tx=0x7fe2a80015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.201+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8040070 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.202+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a80319d0 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.202+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac132710 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.202+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8031da0 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.202+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1421c0 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.203+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac141ff0 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.203+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2a8043070 con 0x7fe2980039e0 2024-05-27T00:39:35.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.204+0000 7fe2a1ffb640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe29006be60 0x7fe29006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.204+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe2a80abc00 con 0x7fe2980039e0 2024-05-27T00:39:35.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.204+0000 7fe2b0905640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe29006be60 0x7fe29006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.204+0000 7fe2b0905640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe29006be60 0x7fe29006e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac15cb10 tx=0x7fe29c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.208+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe2a80b2080 con 0x7fe2980039e0 2024-05-27T00:39:35.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.380+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2ac068680 con 0x7fe2980039e0 2024-05-27T00:39:35.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.381+0000 7fe2a1ffb640 1 -- 172.21.15.111:0/144313460 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fe2a807a300 con 0x7fe2980039e0 2024-05-27T00:39:35.382 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe29006be60 msgr2=0x7fe29006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe29006be60 0x7fe29006e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac15cb10 tx=0x7fe29c008040 comp rx=0 tx=0).stop 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 msgr2=0x7fe2ac15a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe2ac15a200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8002080 tx=0x7fe2a80015b0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 shutdown_connections 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe29006be60 0x7fe29006e320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2ac144d20 0x7fe2ac1451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe2ac132290 0x7fe2ac155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 --2- 172.21.15.111:0/144313460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2980039e0 0x7fe2ac15a200 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 >> 172.21.15.111:0/144313460 conn(0x7fe2ac0b1c00 msgr2=0x7fe2ac0af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 shutdown_connections 2024-05-27T00:39:35.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.383+0000 7fe2b336a640 1 -- 172.21.15.111:0/144313460 wait complete. 2024-05-27T00:39:35.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:39:35.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.495+0000 7f5b754c3640 1 Processor -- start 2024-05-27T00:39:35.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.495+0000 7f5b754c3640 1 -- start start 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b754c3640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5b700c6e20 0x7f5b700b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b754c3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b700cf080 0x7f5b700cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b754c3640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b700c72b0 con 0x7f5b70147120 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b754c3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f5b700c7590 con 0x7f5b700cf080 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b754c3640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5b700c7870 con 0x7f5b700c6e20 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b6f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b700cf080 0x7f5b700cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b6f7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b700cf080 0x7f5b700cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56724/0 (socket says 172.21.15.111:56724) 2024-05-27T00:39:35.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.496+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/368663631 learned_addr learned my addr 172.21.15.111:0/368663631 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:35.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.497+0000 7f5b6e7fc640 1 --2- 172.21.15.111:0/368663631 >> v2:172.21.15.153:3300/0 conn(0x7f5b700c6e20 0x7f5b700b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.497+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/368663631 >> v2:172.21.15.153:3300/0 conn(0x7f5b700c6e20 msgr2=0x7f5b700b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:35.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.497+0000 7f5b6f7fe640 1 --2- 172.21.15.111:0/368663631 >> v2:172.21.15.153:3300/0 conn(0x7f5b700c6e20 0x7f5b700b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.497+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/368663631 >> v1:172.21.15.153:6789/0 conn(0x7f5b70147120 legacy=0x7f5b700ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:35.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.497+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/368663631 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b700c7b50 con 0x7f5b700cf080 2024-05-27T00:39:35.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.497+0000 7f5b6f7fe640 1 --2- 172.21.15.111:0/368663631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b700cf080 0x7f5b700cc5b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009870 tx=0x7f5b6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87b23c6b6e9a4e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b6dffb640 1 -- 172.21.15.111:0/368663631 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68037070 con 0x7f5b700cf080 2024-05-27T00:39:35.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b6dffb640 1 -- 172.21.15.111:0/368663631 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68032440 con 0x7f5b700cf080 2024-05-27T00:39:35.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b6dffb640 1 -- 172.21.15.111:0/368663631 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68032810 con 0x7f5b700cf080 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 -- 172.21.15.111:0/368663631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b700cf080 msgr2=0x7f5b700cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 --2- 172.21.15.111:0/368663631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b700cf080 0x7f5b700cc5b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009870 tx=0x7f5b6802a270 comp rx=0 tx=0).stop 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 -- 172.21.15.111:0/368663631 shutdown_connections 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 --2- 172.21.15.111:0/368663631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b700cf080 0x7f5b700cc5b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 --2- 172.21.15.111:0/368663631 >> v2:172.21.15.153:3300/0 conn(0x7f5b700c6e20 0x7f5b700b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 -- 172.21.15.111:0/368663631 >> 172.21.15.111:0/368663631 conn(0x7f5b700d1f60 msgr2=0x7f5b700d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 -- 172.21.15.111:0/368663631 shutdown_connections 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.498+0000 7f5b754c3640 1 -- 172.21.15.111:0/368663631 wait complete. 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.499+0000 7f5b754c3640 1 Processor -- start 2024-05-27T00:39:35.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.499+0000 7f5b754c3640 1 -- start start 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b754c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b700cf080 0x7f5b70140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b754c3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b70147120 0x7f5b701e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b754c3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 0x7f5b701e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b754c3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701477d0 con 0x7f5b701e3ba0 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b754c3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700c8350 con 0x7f5b700cf080 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b754c3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5b70147950 con 0x7f5b70147120 2024-05-27T00:39:35.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 0x7f5b701e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b6f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 0x7f5b701e54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56728/0 (socket says 172.21.15.111:56728) 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/3995343187 learned_addr learned my addr 172.21.15.111:0/3995343187 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b6e7fc640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b70147120 0x7f5b701e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.500+0000 7f5b6effd640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b700cf080 0x7f5b70140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b70147120 msgr2=0x7f5b701e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6f7fe640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b70147120 0x7f5b701e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b700cf080 msgr2=0x7f5b70140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6f7fe640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b700cf080 0x7f5b70140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6f7fe640 1 -- 172.21.15.111:0/3995343187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b701e9390 con 0x7f5b701e3ba0 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6e7fc640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b70147120 0x7f5b701e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b6f7fe640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 0x7f5b701e54c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b6802a2b0 tx=0x7f5b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68041020 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68030700 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68037040 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b701e9670 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.501+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701e9c30 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.502+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b70068680 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.502+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b68044020 con 0x7f5b701e3ba0 2024-05-27T00:39:35.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.503+0000 7f5b677fe640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b5006be30 0x7f5b5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.503+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5b680ab900 con 0x7f5b701e3ba0 2024-05-27T00:39:35.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.503+0000 7f5b6effd640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b5006be30 0x7f5b5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.503+0000 7f5b6effd640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b5006be30 0x7f5b5006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000b450 tx=0x7f5b6000d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.507+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5b6804b030 con 0x7f5b701e3ba0 2024-05-27T00:39:35.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.684+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5b700d1940 con 0x7f5b701e3ba0 2024-05-27T00:39:35.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.685+0000 7f5b677fe640 1 -- 172.21.15.111:0/3995343187 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f5b68079f70 con 0x7f5b701e3ba0 2024-05-27T00:39:35.686 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:35.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b5006be30 msgr2=0x7f5b5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b5006be30 0x7f5b5006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000b450 tx=0x7f5b6000d040 comp rx=0 tx=0).stop 2024-05-27T00:39:35.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 msgr2=0x7f5b701e54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 0x7f5b701e54c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b6802a2b0 tx=0x7f5b680046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 shutdown_connections 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b5006be30 0x7f5b5006e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b701e3ba0 0x7f5b701e54c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b70147120 0x7f5b701e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 --2- 172.21.15.111:0/3995343187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b700cf080 0x7f5b70140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 >> 172.21.15.111:0/3995343187 conn(0x7f5b700d1f60 msgr2=0x7f5b700cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 shutdown_connections 2024-05-27T00:39:35.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.687+0000 7f5b754c3640 1 -- 172.21.15.111:0/3995343187 wait complete. 2024-05-27T00:39:35.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T00:39:35.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.798+0000 7fbded789640 1 Processor -- start 2024-05-27T00:39:35.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.798+0000 7fbded789640 1 -- start start 2024-05-27T00:39:35.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbded789640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbde80d3320 0x7fbde81622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbded789640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde806cc70 con 0x7fbde8161e50 2024-05-27T00:39:35.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbded789640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde80d1e50 con 0x7fbde80b3630 2024-05-27T00:39:35.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbded789640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fbde80d2130 con 0x7fbde80d3320 2024-05-27T00:39:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbde77fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbde80d3320 0x7fbde81622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbde67fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbde8161e50 0x7fbde80d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35386/0 (socket says 172.21.15.111:35386) 2024-05-27T00:39:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbde67fc640 1 -- 172.21.15.111:0/3878541397 learned_addr learned my addr 172.21.15.111:0/3878541397 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbde77fe640 1 -- 172.21.15.111:0/3878541397 >> v1:172.21.15.153:6789/0 conn(0x7fbde80b3630 legacy=0x7fbde80b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbde77fe640 1 -- 172.21.15.111:0/3878541397 >> v1:172.21.15.111:6789/0 conn(0x7fbde8161e50 legacy=0x7fbde80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.799+0000 7fbde77fe640 1 -- 172.21.15.111:0/3878541397 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde80d2410 con 0x7fbde80d3320 2024-05-27T00:39:35.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.800+0000 7fbde77fe640 1 --2- 172.21.15.111:0/3878541397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde81622e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1efd10f077373d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.800+0000 7fbde5ffb640 1 -- 172.21.15.111:0/3878541397 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0007740 con 0x7fbde80d3320 2024-05-27T00:39:35.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.800+0000 7fbde5ffb640 1 -- 172.21.15.111:0/3878541397 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde00078e0 con 0x7fbde80d3320 2024-05-27T00:39:35.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.800+0000 7fbde5ffb640 1 -- 172.21.15.111:0/3878541397 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0007cb0 con 0x7fbde80d3320 2024-05-27T00:39:35.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 -- 172.21.15.111:0/3878541397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 msgr2=0x7fbde81622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 --2- 172.21.15.111:0/3878541397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde81622e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde002a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 -- 172.21.15.111:0/3878541397 shutdown_connections 2024-05-27T00:39:35.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 --2- 172.21.15.111:0/3878541397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde81622e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 -- 172.21.15.111:0/3878541397 >> 172.21.15.111:0/3878541397 conn(0x7fbde80d42c0 msgr2=0x7fbde80d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 -- 172.21.15.111:0/3878541397 shutdown_connections 2024-05-27T00:39:35.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.801+0000 7fbded789640 1 -- 172.21.15.111:0/3878541397 wait complete. 2024-05-27T00:39:35.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.802+0000 7fbded789640 1 Processor -- start 2024-05-27T00:39:35.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.802+0000 7fbded789640 1 -- start start 2024-05-27T00:39:35.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbded789640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde80b3630 0x7fbde81f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbded789640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbded789640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbde8161e50 0x7fbde80c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbded789640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbde806bfb0 con 0x7fbde80d3320 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbded789640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbde80b79b0 con 0x7fbde80b3630 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbded789640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbde806c130 con 0x7fbde8161e50 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde815ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56740/0 (socket says 172.21.15.111:56740) 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 -- 172.21.15.111:0/3468831921 learned_addr learned my addr 172.21.15.111:0/3468831921 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde6ffd640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde80b3630 0x7fbde81f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde77fe640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbde8161e50 0x7fbde80c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 -- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbde8161e50 msgr2=0x7fbde80c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbde8161e50 0x7fbde80c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 -- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde80b3630 msgr2=0x7fbde81f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde80b3630 0x7fbde81f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 -- 172.21.15.111:0/3468831921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81e9340 con 0x7fbde80d3320 2024-05-27T00:39:35.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde6ffd640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde80b3630 0x7fbde81f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:35.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.803+0000 7fbde67fc640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde815ce50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00c3c0 tx=0x7fbddc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.804+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc017070 con 0x7fbde80d3320 2024-05-27T00:39:35.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.804+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc005ce0 con 0x7fbde80d3320 2024-05-27T00:39:35.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.804+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde80c8c40 con 0x7fbde80d3320 2024-05-27T00:39:35.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.804+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0072f0 con 0x7fbde80d3320 2024-05-27T00:39:35.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.804+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde80c9200 con 0x7fbde80d3320 2024-05-27T00:39:35.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.805+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde8068680 con 0x7fbde80d3320 2024-05-27T00:39:35.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.805+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbddc01a070 con 0x7fbde80d3320 2024-05-27T00:39:35.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.806+0000 7fbdcf7fe640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbdd406be60 0x7fbdd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:35.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.806+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbddc08a590 con 0x7fbde80d3320 2024-05-27T00:39:35.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.806+0000 7fbde6ffd640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbdd406be60 0x7fbdd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:35.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.806+0000 7fbde6ffd640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbdd406be60 0x7fbdd406e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbde80b7960 tx=0x7fbdd8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:35.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.810+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbddc058c00 con 0x7fbde80d3320 2024-05-27T00:39:35.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.993+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbde81f4620 con 0x7fbde80d3320 2024-05-27T00:39:35.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.994+0000 7fbdcf7fe640 1 -- 172.21.15.111:0/3468831921 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fbddc058500 con 0x7fbde80d3320 2024-05-27T00:39:35.995 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbdd406be60 msgr2=0x7fbdd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbdd406be60 0x7fbdd406e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbde80b7960 tx=0x7fbdd8009040 comp rx=0 tx=0).stop 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 msgr2=0x7fbde815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde815ce50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00c3c0 tx=0x7fbddc00c880 comp rx=0 tx=0).stop 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 shutdown_connections 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbdd406be60 0x7fbdd406e320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbde8161e50 0x7fbde80c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbde80d3320 0x7fbde815ce50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 --2- 172.21.15.111:0/3468831921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbde80b3630 0x7fbde81f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.995+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 >> 172.21.15.111:0/3468831921 conn(0x7fbde80d42c0 msgr2=0x7fbde80d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.996+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 shutdown_connections 2024-05-27T00:39:35.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:35.996+0000 7fbded789640 1 -- 172.21.15.111:0/3468831921 wait complete. 2024-05-27T00:39:36.007 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2024-05-27T00:39:36.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:36.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-27T00:39:36.031 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[44991]: fuse finished with error 0 and tester_r 0 2024-05-27T00:39:36.032 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T00:39:36.115 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:39:36.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:36.115 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-27T00:39:36.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:36.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.273+0000 7fa2c095a640 1 Processor -- start 2024-05-27T00:39:36.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.273+0000 7fa2c095a640 1 -- start start 2024-05-27T00:39:36.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2c095a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2c095a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2bc0726d0 0x7fa2bc072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2c095a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2bc0b3440 con 0x7fa2bc06b390 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2c095a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa2bc073080 con 0x7fa2bc0726d0 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2c095a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc0b42f0 con 0x7fa2bc144000 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2bad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa2bc0726d0 0x7fa2bc072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2b3fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa2bc06b390 0x7fa2bc0cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35392/0 (socket says 172.21.15.111:35392) 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2b3fff640 1 -- 172.21.15.111:0/293698027 learned_addr learned my addr 172.21.15.111:0/293698027 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2ba59c640 1 --2- 172.21.15.111:0/293698027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2bad9d640 1 -- 172.21.15.111:0/293698027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 msgr2=0x7fa2bc0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2bad9d640 1 --2- 172.21.15.111:0/293698027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2bad9d640 1 -- 172.21.15.111:0/293698027 >> v1:172.21.15.111:6789/0 conn(0x7fa2bc06b390 legacy=0x7fa2bc0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2b9d9b640 1 -- 172.21.15.111:0/293698027 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155482406 0 0) 0x7fa2bc0b3440 con 0x7fa2bc06b390 2024-05-27T00:39:36.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.274+0000 7fa2bad9d640 1 -- 172.21.15.111:0/293698027 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc0b36e0 con 0x7fa2bc0726d0 2024-05-27T00:39:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.275+0000 7fa2bad9d640 1 --2- 172.21.15.111:0/293698027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc072b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d90 tx=0x7fa2b402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275188959676a758 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.275+0000 7fa2b9d9b640 1 -- 172.21.15.111:0/293698027 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4007740 con 0x7fa2bc0726d0 2024-05-27T00:39:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.275+0000 7fa2b9d9b640 1 -- 172.21.15.111:0/293698027 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b40078e0 con 0x7fa2bc0726d0 2024-05-27T00:39:36.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.275+0000 7fa2b9d9b640 1 -- 172.21.15.111:0/293698027 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4007cb0 con 0x7fa2bc0726d0 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 -- 172.21.15.111:0/293698027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 msgr2=0x7fa2bc072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 --2- 172.21.15.111:0/293698027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc072b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d90 tx=0x7fa2b402a660 comp rx=0 tx=0).stop 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 -- 172.21.15.111:0/293698027 shutdown_connections 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 --2- 172.21.15.111:0/293698027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc072b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 --2- 172.21.15.111:0/293698027 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 -- 172.21.15.111:0/293698027 >> 172.21.15.111:0/293698027 conn(0x7fa2bc0b1c00 msgr2=0x7fa2bc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 -- 172.21.15.111:0/293698027 shutdown_connections 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.276+0000 7fa2c095a640 1 -- 172.21.15.111:0/293698027 wait complete. 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.277+0000 7fa2c095a640 1 Processor -- start 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.277+0000 7fa2c095a640 1 -- start start 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2c095a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2bc06b390 0x7fa2bc13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2c095a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc1f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2c095a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2c095a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc0b3f80 con 0x7fa2bc0726d0 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2c095a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc0b3ca0 con 0x7fa2bc06b390 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2c095a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa2bc0cde50 con 0x7fa2bc144000 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc1f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc1f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56762/0 (socket says 172.21.15.111:56762) 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 -- 172.21.15.111:0/1735531698 learned_addr learned my addr 172.21.15.111:0/1735531698 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2bad9d640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2ba59c640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2bc06b390 0x7fa2bc13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 -- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 msgr2=0x7fa2bc140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 -- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2bc06b390 msgr2=0x7fa2bc13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2bc06b390 0x7fa2bc13eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 -- 172.21.15.111:0/1735531698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc1f10a0 con 0x7fa2bc0726d0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b3fff640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc1f05b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e780 tx=0x7fa2a400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40072e0 con 0x7fa2bc0726d0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.278+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a4016020 con 0x7fa2bc0726d0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.279+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc1f1380 con 0x7fa2bc0726d0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.279+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400c7f0 con 0x7fa2bc0726d0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.279+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1f1940 con 0x7fa2bc0726d0 2024-05-27T00:39:36.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.280+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc1f1b40 con 0x7fa2bc0726d0 2024-05-27T00:39:36.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.280+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2a400c990 con 0x7fa2bc0726d0 2024-05-27T00:39:36.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.280+0000 7fa2b2ffd640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa29c06be30 0x7fa29c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.281+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa2a408d310 con 0x7fa2bc0726d0 2024-05-27T00:39:36.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.281+0000 7fa2ba59c640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa29c06be30 0x7fa29c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.281+0000 7fa2ba59c640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa29c06be30 0x7fa29c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0079a0 tx=0x7fa2ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.285+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa2bc1f1b40 con 0x7fa2bc0726d0 2024-05-27T00:39:36.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.457+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2bc068680 con 0x7fa2bc0726d0 2024-05-27T00:39:36.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.458+0000 7fa2b2ffd640 1 -- 172.21.15.111:0/1735531698 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fa2a4020030 con 0x7fa2bc0726d0 2024-05-27T00:39:36.459 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:36.460 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:36.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa29c06be30 msgr2=0x7fa29c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa29c06be30 0x7fa29c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0079a0 tx=0x7fa2ac008040 comp rx=0 tx=0).stop 2024-05-27T00:39:36.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 msgr2=0x7fa2bc1f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc1f05b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e780 tx=0x7fa2a400ec40 comp rx=0 tx=0).stop 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 shutdown_connections 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa29c06be30 0x7fa29c06e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa2bc144000 0x7fa2bc140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0726d0 0x7fa2bc1f05b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 --2- 172.21.15.111:0/1735531698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2bc06b390 0x7fa2bc13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 >> 172.21.15.111:0/1735531698 conn(0x7fa2bc0b1c00 msgr2=0x7fa2bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 shutdown_connections 2024-05-27T00:39:36.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.460+0000 7fa2c095a640 1 -- 172.21.15.111:0/1735531698 wait complete. 2024-05-27T00:39:36.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:36.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.571+0000 7fd1dd831640 1 Processor -- start 2024-05-27T00:39:36.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.571+0000 7fd1dd831640 1 -- start start 2024-05-27T00:39:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1dd831640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd1d8147120 0x7fd1d80b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1dd831640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd1d80c6e20 0x7fd1d80ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1dd831640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d80c72b0 con 0x7fd1d80ceeb0 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1dd831640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d80c7590 con 0x7fd1d8147120 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1dd831640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d80c7870 con 0x7fd1d80c6e20 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d6ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd1d8147120 0x7fd1d80b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d67fc640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd1d80c6e20 0x7fd1d80ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd1d80ceeb0 0x7fd1d8161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35400/0 (socket says 172.21.15.111:35400) 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d77fe640 1 -- 172.21.15.111:0/3083802639 learned_addr learned my addr 172.21.15.111:0/3083802639 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d6ffd640 1 -- 172.21.15.111:0/3083802639 >> v2:172.21.15.153:3300/0 conn(0x7fd1d80c6e20 msgr2=0x7fd1d80ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d6ffd640 1 --2- 172.21.15.111:0/3083802639 >> v2:172.21.15.153:3300/0 conn(0x7fd1d80c6e20 0x7fd1d80ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.572+0000 7fd1d6ffd640 1 -- 172.21.15.111:0/3083802639 >> v1:172.21.15.111:6789/0 conn(0x7fd1d80ceeb0 legacy=0x7fd1d8161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.573+0000 7fd1d6ffd640 1 -- 172.21.15.111:0/3083802639 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d80c7b50 con 0x7fd1d8147120 2024-05-27T00:39:36.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.573+0000 7fd1d6ffd640 1 --2- 172.21.15.111:0/3083802639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d80b3360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b2e70 tx=0x7fd1c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcaa6146f933ee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.573+0000 7fd1d5ffb640 1 -- 172.21.15.111:0/3083802639 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8037070 con 0x7fd1d8147120 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.573+0000 7fd1d5ffb640 1 -- 172.21.15.111:0/3083802639 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c8032440 con 0x7fd1d8147120 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.573+0000 7fd1d5ffb640 1 -- 172.21.15.111:0/3083802639 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8032810 con 0x7fd1d8147120 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 -- 172.21.15.111:0/3083802639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 msgr2=0x7fd1d80b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 --2- 172.21.15.111:0/3083802639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d80b3360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b2e70 tx=0x7fd1c802a1a0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 -- 172.21.15.111:0/3083802639 shutdown_connections 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 --2- 172.21.15.111:0/3083802639 >> v2:172.21.15.153:3300/0 conn(0x7fd1d80c6e20 0x7fd1d80ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 --2- 172.21.15.111:0/3083802639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d80b3360 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 -- 172.21.15.111:0/3083802639 >> 172.21.15.111:0/3083802639 conn(0x7fd1d80d1f60 msgr2=0x7fd1d80d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 -- 172.21.15.111:0/3083802639 shutdown_connections 2024-05-27T00:39:36.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.574+0000 7fd1dd831640 1 -- 172.21.15.111:0/3083802639 wait complete. 2024-05-27T00:39:36.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.575+0000 7fd1dd831640 1 Processor -- start 2024-05-27T00:39:36.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.575+0000 7fd1dd831640 1 -- start start 2024-05-27T00:39:36.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1dd831640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1d80ceeb0 0x7fd1d81f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1dd831640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1dd831640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1d80b1190 0x7fd1d80cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1dd831640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d80cd5c0 con 0x7fd1d8147120 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1dd831640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d80c8350 con 0x7fd1d80b1190 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1dd831640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d81477d0 con 0x7fd1d80ceeb0 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d813bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56776/0 (socket says 172.21.15.111:56776) 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1d67fc640 1 -- 172.21.15.111:0/1530894734 learned_addr learned my addr 172.21.15.111:0/1530894734 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1d6ffd640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1d80ceeb0 0x7fd1d81f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.576+0000 7fd1d77fe640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1d80b1190 0x7fd1d80cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1d67fc640 1 -- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1d80ceeb0 msgr2=0x7fd1d81f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1d67fc640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1d80ceeb0 0x7fd1d81f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1d67fc640 1 -- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1d80b1190 msgr2=0x7fd1d80cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1d67fc640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1d80b1190 0x7fd1d80cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1d67fc640 1 -- 172.21.15.111:0/1530894734 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d813c0a0 con 0x7fd1d8147120 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1d67fc640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d813bb60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00e870 tx=0x7fd1cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc017070 con 0x7fd1d8147120 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1cc00c8e0 con 0x7fd1d8147120 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00ccd0 con 0x7fd1d8147120 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d80b16e0 con 0x7fd1d8147120 2024-05-27T00:39:36.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.577+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d80b1ca0 con 0x7fd1d8147120 2024-05-27T00:39:36.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.578+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d8073890 con 0x7fd1d8147120 2024-05-27T00:39:36.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.578+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1cc01a070 con 0x7fd1d8147120 2024-05-27T00:39:36.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.579+0000 7fd1bffff640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1c406be30 0x7fd1c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.579+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd1cc08ae90 con 0x7fd1d8147120 2024-05-27T00:39:36.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.579+0000 7fd1d6ffd640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1c406be30 0x7fd1c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.580+0000 7fd1d6ffd640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1c406be30 0x7fd1c406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8034000 tx=0x7fd1c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.584+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd1d8073890 con 0x7fd1d8147120 2024-05-27T00:39:36.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.759+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1d8068680 con 0x7fd1d8147120 2024-05-27T00:39:36.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.760+0000 7fd1bffff640 1 -- 172.21.15.111:0/1530894734 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fd1cc058f40 con 0x7fd1d8147120 2024-05-27T00:39:36.761 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:36.761 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1c406be30 msgr2=0x7fd1c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1c406be30 0x7fd1c406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8034000 tx=0x7fd1c802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 msgr2=0x7fd1d813bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d813bb60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00e870 tx=0x7fd1cc00ed30 comp rx=0 tx=0).stop 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 shutdown_connections 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd1c406be30 0x7fd1c406e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1d80b1190 0x7fd1d80cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d8147120 0x7fd1d813bb60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 --2- 172.21.15.111:0/1530894734 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd1d80ceeb0 0x7fd1d81f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 >> 172.21.15.111:0/1530894734 conn(0x7fd1d80d1f60 msgr2=0x7fd1d806f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:36.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 shutdown_connections 2024-05-27T00:39:36.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.762+0000 7fd1dd831640 1 -- 172.21.15.111:0/1530894734 wait complete. 2024-05-27T00:39:36.774 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-27T00:39:36.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.876+0000 7f48caf5a640 1 Processor -- start 2024-05-27T00:39:36.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.876+0000 7f48caf5a640 1 -- start start 2024-05-27T00:39:36.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48caf5a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48c415c610 0x7f48c415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48caf5a640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f48c406fd90 0x7f48c415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48caf5a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c415d470 con 0x7f48c40b3ed0 2024-05-27T00:39:36.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48caf5a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f48c40ce240 con 0x7f48c415c610 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48caf5a640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f48c40b3a40 con 0x7f48c406fd90 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c8cf6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48c415c610 0x7f48c415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c94f7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f48c40b3ed0 0x7f48c40b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35404/0 (socket says 172.21.15.111:35404) 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c8cf6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f48c415c610 0x7f48c415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56782/0 (socket says 172.21.15.111:56782) 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c94f7640 1 -- 172.21.15.111:0/3121950815 learned_addr learned my addr 172.21.15.111:0/3121950815 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c8cf6640 1 -- 172.21.15.111:0/3121950815 >> v2:172.21.15.153:3300/0 conn(0x7f48c406fd90 msgr2=0x7f48c415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c8cf6640 1 --2- 172.21.15.111:0/3121950815 >> v2:172.21.15.153:3300/0 conn(0x7f48c406fd90 0x7f48c415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c8cf6640 1 -- 172.21.15.111:0/3121950815 >> v1:172.21.15.111:6789/0 conn(0x7f48c40b3ed0 legacy=0x7f48c40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c8cf6640 1 -- 172.21.15.111:0/3121950815 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c40ce470 con 0x7f48c415c610 2024-05-27T00:39:36.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.877+0000 7f48c37fe640 1 -- 172.21.15.111:0/3121950815 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353538993 0 0) 0x7f48bc001490 con 0x7f48c40b3ed0 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.878+0000 7f48c8cf6640 1 --2- 172.21.15.111:0/3121950815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c415c9f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f48c40cfcd0 tx=0x7f48b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc098a5e7cadb38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.878+0000 7f48c37fe640 1 -- 172.21.15.111:0/3121950815 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8037070 con 0x7f48c415c610 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.878+0000 7f48c37fe640 1 -- 172.21.15.111:0/3121950815 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b8032440 con 0x7f48c415c610 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.878+0000 7f48c37fe640 1 -- 172.21.15.111:0/3121950815 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b802ae60 con 0x7f48c415c610 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 -- 172.21.15.111:0/3121950815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 msgr2=0x7f48c415c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 --2- 172.21.15.111:0/3121950815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c415c9f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f48c40cfcd0 tx=0x7f48b802a110 comp rx=0 tx=0).stop 2024-05-27T00:39:36.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 -- 172.21.15.111:0/3121950815 shutdown_connections 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 --2- 172.21.15.111:0/3121950815 >> v2:172.21.15.153:3300/0 conn(0x7f48c406fd90 0x7f48c415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 --2- 172.21.15.111:0/3121950815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c415c9f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 -- 172.21.15.111:0/3121950815 >> 172.21.15.111:0/3121950815 conn(0x7f48c4072a30 msgr2=0x7f48c406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 -- 172.21.15.111:0/3121950815 shutdown_connections 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.879+0000 7f48caf5a640 1 -- 172.21.15.111:0/3121950815 wait complete. 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.880+0000 7f48caf5a640 1 Processor -- start 2024-05-27T00:39:36.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.880+0000 7f48caf5a640 1 -- start start 2024-05-27T00:39:36.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48c406fd90 0x7f48c4159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48c40b3ed0 0x7f48c413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c413f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40ced10 con 0x7f48c415c610 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40cea30 con 0x7f48c40b3ed0 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f48c415c300 con 0x7f48c406fd90 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c413f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c413f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56784/0 (socket says 172.21.15.111:56784) 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 -- 172.21.15.111:0/676134217 learned_addr learned my addr 172.21.15.111:0/676134217 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:36.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c3fff640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48c40b3ed0 0x7f48c413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c8cf6640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48c406fd90 0x7f48c4159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 -- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48c406fd90 msgr2=0x7f48c4159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48c406fd90 0x7f48c4159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 -- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48c40b3ed0 msgr2=0x7f48c413ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48c40b3ed0 0x7f48c413ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 -- 172.21.15.111:0/676134217 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c4155e80 con 0x7f48c415c610 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c8cf6640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48c406fd90 0x7f48c4159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c94f7640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c413f190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00c700 tx=0x7f48bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc018070 con 0x7f48c415c610 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc00ce10 con 0x7f48c415c610 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c4156160 con 0x7f48c415c610 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.881+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00a860 con 0x7f48c415c610 2024-05-27T00:39:36.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.882+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c4156720 con 0x7f48c415c610 2024-05-27T00:39:36.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.883+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c4068680 con 0x7f48c415c610 2024-05-27T00:39:36.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.883+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48bc01b070 con 0x7f48c415c610 2024-05-27T00:39:36.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.883+0000 7f48c17fa640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f48a806be60 0x7f48a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:36.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.883+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48bc08a6e0 con 0x7f48c415c610 2024-05-27T00:39:36.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.884+0000 7f48c8cf6640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f48a806be60 0x7f48a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:36.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.884+0000 7f48c8cf6640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f48a806be60 0x7f48a806e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48b8034000 tx=0x7f48b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:36.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:36.888+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48c415d470 con 0x7f48c415c610 2024-05-27T00:39:37.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.102+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f48c4074cf0 con 0x7f48c415c610 2024-05-27T00:39:37.632 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.632+0000 7f48c17fa640 1 -- 172.21.15.111:0/676134217 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f48bc058700 con 0x7f48c415c610 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f48a806be60 msgr2=0x7f48a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f48a806be60 0x7f48a806e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48b8034000 tx=0x7f48b802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 msgr2=0x7f48c413f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c413f190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00c700 tx=0x7f48bc00cbc0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 shutdown_connections 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f48a806be60 0x7f48a806e320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.635 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c415c610 0x7f48c413f190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48c40b3ed0 0x7f48c413ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 --2- 172.21.15.111:0/676134217 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48c406fd90 0x7f48c4159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 >> 172.21.15.111:0/676134217 conn(0x7f48c4072a30 msgr2=0x7f48c406ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:37.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 shutdown_connections 2024-05-27T00:39:37.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.634+0000 7f48caf5a640 1 -- 172.21.15.111:0/676134217 wait complete. 2024-05-27T00:39:37.647 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-27T00:39:37.647 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-27T00:39:37.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.750+0000 7f1f3994f640 1 Processor -- start 2024-05-27T00:39:37.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.750+0000 7f1f3994f640 1 -- start start 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.750+0000 7f1f3994f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f34161e50 0x7f1f341622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.750+0000 7f1f3994f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f3406cc70 con 0x7f1f340d2c10 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.750+0000 7f1f3994f640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f340d1e50 con 0x7f1f340b3630 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.750+0000 7f1f3994f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1f340d2130 con 0x7f1f34161e50 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f327fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f34161e50 0x7f1f341622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f337fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1f340d2c10 0x7f1f340d3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35418/0 (socket says 172.21.15.111:35418) 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f337fe640 1 -- 172.21.15.111:0/3038809822 learned_addr learned my addr 172.21.15.111:0/3038809822 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f327fc640 1 -- 172.21.15.111:0/3038809822 >> v1:172.21.15.153:6789/0 conn(0x7f1f340b3630 legacy=0x7f1f340b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:37.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f31ffb640 1 -- 172.21.15.111:0/3038809822 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495891137 0 0) 0x7f1f3406cc70 con 0x7f1f340d2c10 2024-05-27T00:39:37.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f327fc640 1 -- 172.21.15.111:0/3038809822 >> v1:172.21.15.111:6789/0 conn(0x7f1f340d2c10 legacy=0x7f1f340d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:37.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.751+0000 7f1f327fc640 1 -- 172.21.15.111:0/3038809822 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f340d2410 con 0x7f1f34161e50 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f327fc640 1 --2- 172.21.15.111:0/3038809822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f341622e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f280097a0 tx=0x7f1f2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92ede4d2351081c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f31ffb640 1 -- 172.21.15.111:0/3038809822 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28037070 con 0x7f1f34161e50 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f31ffb640 1 -- 172.21.15.111:0/3038809822 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f28032440 con 0x7f1f34161e50 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f31ffb640 1 -- 172.21.15.111:0/3038809822 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28032810 con 0x7f1f34161e50 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 -- 172.21.15.111:0/3038809822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 msgr2=0x7f1f341622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 --2- 172.21.15.111:0/3038809822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f341622e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f280097a0 tx=0x7f1f2802a1e0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 -- 172.21.15.111:0/3038809822 shutdown_connections 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 --2- 172.21.15.111:0/3038809822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f341622e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 -- 172.21.15.111:0/3038809822 >> 172.21.15.111:0/3038809822 conn(0x7f1f340d42c0 msgr2=0x7f1f340d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 -- 172.21.15.111:0/3038809822 shutdown_connections 2024-05-27T00:39:37.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.752+0000 7f1f3994f640 1 -- 172.21.15.111:0/3038809822 wait complete. 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.753+0000 7f1f3994f640 1 Processor -- start 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 -- start start 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b3630 0x7f1f3415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340d2c10 0x7f1f340cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f340cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3406bfb0 con 0x7f1f34161e50 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f340b79b0 con 0x7f1f340d2c10 2024-05-27T00:39:37.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f3994f640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3406c130 con 0x7f1f340b3630 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f337fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f340cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f337fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f340cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56806/0 (socket says 172.21.15.111:56806) 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.754+0000 7f1f337fe640 1 -- 172.21.15.111:0/2952630260 learned_addr learned my addr 172.21.15.111:0/2952630260 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f327fc640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340d2c10 0x7f1f340cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f32ffd640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b3630 0x7f1f3415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f337fe640 1 -- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b3630 msgr2=0x7f1f3415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f337fe640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b3630 0x7f1f3415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f337fe640 1 -- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340d2c10 msgr2=0x7f1f340cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f337fe640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340d2c10 0x7f1f340cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f337fe640 1 -- 172.21.15.111:0/2952630260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f340cc7e0 con 0x7f1f34161e50 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f327fc640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340d2c10 0x7f1f340cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f337fe640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f340cf360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c7a0 tx=0x7f1f2c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:37.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c018070 con 0x7f1f34161e50 2024-05-27T00:39:37.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c00a640 con 0x7f1f34161e50 2024-05-27T00:39:37.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c00aab0 con 0x7f1f34161e50 2024-05-27T00:39:37.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f340cbf40 con 0x7f1f34161e50 2024-05-27T00:39:37.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.755+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f340cc500 con 0x7f1f34161e50 2024-05-27T00:39:37.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.756+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f34068680 con 0x7f1f34161e50 2024-05-27T00:39:37.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.756+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f2c00ac50 con 0x7f1f34161e50 2024-05-27T00:39:37.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.757+0000 7f1f1bfff640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f2006be30 0x7f1f2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:37.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.757+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1f2c08a6f0 con 0x7f1f34161e50 2024-05-27T00:39:37.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.757+0000 7f1f32ffd640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f2006be30 0x7f1f2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:37.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.758+0000 7f1f32ffd640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f2006be30 0x7f1f2006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f340b75f0 tx=0x7f1f24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:37.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.761+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1f2c058df0 con 0x7f1f34161e50 2024-05-27T00:39:37.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:37.970+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1f3406ce90 con 0x7f1f34161e50 2024-05-27T00:39:38.634 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.633+0000 7f1f1bfff640 1 -- 172.21.15.111:0/2952630260 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f1f2c0586f0 con 0x7f1f34161e50 2024-05-27T00:39:38.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f2006be30 msgr2=0x7f1f2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f2006be30 0x7f1f2006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f340b75f0 tx=0x7f1f24009040 comp rx=0 tx=0).stop 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 msgr2=0x7f1f340cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f340cf360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c7a0 tx=0x7f1f2c00cc60 comp rx=0 tx=0).stop 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 shutdown_connections 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f2006be30 0x7f1f2006e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f34161e50 0x7f1f340cf360 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f340d2c10 0x7f1f340cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 --2- 172.21.15.111:0/2952630260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f340b3630 0x7f1f3415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 >> 172.21.15.111:0/2952630260 conn(0x7f1f340d42c0 msgr2=0x7f1f340d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.636+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 shutdown_connections 2024-05-27T00:39:38.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.637+0000 7f1f3994f640 1 -- 172.21.15.111:0/2952630260 wait complete. 2024-05-27T00:39:38.650 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-27T00:39:38.651 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:38.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.752+0000 7f2c2b1a2640 1 Processor -- start 2024-05-27T00:39:38.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.752+0000 7f2c2b1a2640 1 -- start start 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c2b1a2640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2c24144000 0x7f2c240b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c2b1a2640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2c240726d0 0x7f2c24072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c2b1a2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c24073020 con 0x7f2c2406b390 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c2b1a2640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f2c240b42f0 con 0x7f2c24144000 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c2b1a2640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2c240b3440 con 0x7f2c240726d0 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c28f3e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2c24144000 0x7f2c240b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c1bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2c2406b390 0x7f2c240cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35420/0 (socket says 172.21.15.111:35420) 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c28f3e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2c24144000 0x7f2c240b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56818/0 (socket says 172.21.15.111:56818) 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c1bfff640 1 -- 172.21.15.111:0/233489526 learned_addr learned my addr 172.21.15.111:0/233489526 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:38.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c2973f640 1 --2- 172.21.15.111:0/233489526 >> v2:172.21.15.153:3300/0 conn(0x7f2c240726d0 0x7f2c24072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c28f3e640 1 -- 172.21.15.111:0/233489526 >> v2:172.21.15.153:3300/0 conn(0x7f2c240726d0 msgr2=0x7f2c24072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c28f3e640 1 --2- 172.21.15.111:0/233489526 >> v2:172.21.15.153:3300/0 conn(0x7f2c240726d0 0x7f2c24072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c28f3e640 1 -- 172.21.15.111:0/233489526 >> v1:172.21.15.111:6789/0 conn(0x7f2c2406b390 legacy=0x7f2c240cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c1b7fe640 1 -- 172.21.15.111:0/233489526 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057656069 0 0) 0x7f2c14001110 con 0x7f2c2406b390 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.753+0000 7f2c28f3e640 1 -- 172.21.15.111:0/233489526 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c240b36e0 con 0x7f2c24144000 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.754+0000 7f2c28f3e640 1 --2- 172.21.15.111:0/233489526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c24144000 0x7f2c240b2f00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c240b2ca0 tx=0x7f2c2002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf43a00184706a06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.754+0000 7f2c1b7fe640 1 -- 172.21.15.111:0/233489526 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20032770 con 0x7f2c24144000 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.754+0000 7f2c1b7fe640 1 -- 172.21.15.111:0/233489526 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c20038070 con 0x7f2c24144000 2024-05-27T00:39:38.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.754+0000 7f2c1b7fe640 1 -- 172.21.15.111:0/233489526 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20032b30 con 0x7f2c24144000 2024-05-27T00:39:38.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/233489526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c24144000 msgr2=0x7f2c240b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/233489526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c24144000 0x7f2c240b2f00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c240b2ca0 tx=0x7f2c2002c870 comp rx=0 tx=0).stop 2024-05-27T00:39:38.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/233489526 shutdown_connections 2024-05-27T00:39:38.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/233489526 >> v2:172.21.15.153:3300/0 conn(0x7f2c240726d0 0x7f2c24072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/233489526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c24144000 0x7f2c240b2f00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/233489526 >> 172.21.15.111:0/233489526 conn(0x7f2c240b1c00 msgr2=0x7f2c240b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:38.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/233489526 shutdown_connections 2024-05-27T00:39:38.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.755+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/233489526 wait complete. 2024-05-27T00:39:38.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.756+0000 7f2c2b1a2640 1 Processor -- start 2024-05-27T00:39:38.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.756+0000 7f2c2b1a2640 1 -- start start 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c2b1a2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 0x7f2c2413eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c2b1a2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c240726d0 0x7f2c2413cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c2b1a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c24144000 0x7f2c240aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c2b1a2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2c240b3f80 con 0x7f2c2406b390 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c2b1a2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c240b3ca0 con 0x7f2c24144000 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c2b1a2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2c240cde50 con 0x7f2c240726d0 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 0x7f2c2413eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 0x7f2c2413eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56828/0 (socket says 172.21.15.111:56828) 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 -- 172.21.15.111:0/1550370857 learned_addr learned my addr 172.21.15.111:0/1550370857 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:38.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 -- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c240726d0 msgr2=0x7f2c2413cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c240726d0 0x7f2c2413cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 -- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c24144000 msgr2=0x7f2c240aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c24144000 0x7f2c240aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 -- 172.21.15.111:0/1550370857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2413fa80 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.757+0000 7f2c28f3e640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 0x7f2c2413eb50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.758+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2004a070 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.758+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c20032e40 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.758+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2413f1e0 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.758+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20033850 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.758+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2413f7a0 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.759+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c24068ef0 con 0x7f2c2406b390 2024-05-27T00:39:38.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.759+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c2004d070 con 0x7f2c2406b390 2024-05-27T00:39:38.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.760+0000 7f2c197fa640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c1006be30 0x7f2c1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:38.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.760+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c200ad8a0 con 0x7f2c2406b390 2024-05-27T00:39:38.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.760+0000 7f2c1bfff640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c1006be30 0x7f2c1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:38.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.761+0000 7f2c1bfff640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c1006be30 0x7f2c1006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c140079c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:38.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.764+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2c24068ef0 con 0x7f2c2406b390 2024-05-27T00:39:38.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.973+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c240af340 con 0x7f2c2406b390 2024-05-27T00:39:38.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.973+0000 7f2c197fa640 1 -- 172.21.15.111:0/1550370857 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f2c2007c0c0 con 0x7f2c2406b390 2024-05-27T00:39:38.974 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:38.975 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:39:38.975 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c1006be30 msgr2=0x7f2c1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c1006be30 0x7f2c1006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c140079c0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 msgr2=0x7f2c2413eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 0x7f2c2413eb50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c200046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 shutdown_connections 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c1006be30 0x7f2c1006e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c24144000 0x7f2c240aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c240726d0 0x7f2c2413cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 --2- 172.21.15.111:0/1550370857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2406b390 0x7f2c2413eb50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:38.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 >> 172.21.15.111:0/1550370857 conn(0x7f2c240b1c00 msgr2=0x7f2c2406b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:38.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 shutdown_connections 2024-05-27T00:39:38.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:38.975+0000 7f2c2b1a2640 1 -- 172.21.15.111:0/1550370857 wait complete. 2024-05-27T00:39:38.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:39.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.086+0000 7fc47be41640 1 Processor -- start 2024-05-27T00:39:39.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.086+0000 7fc47be41640 1 -- start start 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47be41640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474151740 0x7fc474071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47be41640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc474071b40 0x7fc47406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47be41640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc474071fe0 con 0x7fc47406c160 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47be41640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc47406c5d0 con 0x7fc474071b40 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47be41640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc4740722c0 con 0x7fc474151740 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc4793dc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc474071b40 0x7fc47406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47a3de640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc47406c160 0x7fc4740cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35430/0 (socket says 172.21.15.111:35430) 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc4793dc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc474071b40 0x7fc47406bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56838/0 (socket says 172.21.15.111:56838) 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc47a3de640 1 -- 172.21.15.111:0/2076711187 learned_addr learned my addr 172.21.15.111:0/2076711187 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:39.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc479bdd640 1 --2- 172.21.15.111:0/2076711187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474151740 0x7fc474071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc4793dc640 1 -- 172.21.15.111:0/2076711187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474151740 msgr2=0x7fc474071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:39.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc4793dc640 1 --2- 172.21.15.111:0/2076711187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474151740 0x7fc474071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc4793dc640 1 -- 172.21.15.111:0/2076711187 >> v1:172.21.15.111:6789/0 conn(0x7fc47406c160 legacy=0x7fc4740cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.087+0000 7fc4793dc640 1 -- 172.21.15.111:0/2076711187 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4740725a0 con 0x7fc474071b40 2024-05-27T00:39:39.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.088+0000 7fc4793dc640 1 --2- 172.21.15.111:0/2076711187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474071b40 0x7fc47406bc20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc4640097a0 tx=0x7fc46402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ce0e98ed08e687 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.088+0000 7fc478bdb640 1 -- 172.21.15.111:0/2076711187 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464037070 con 0x7fc474071b40 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.088+0000 7fc478bdb640 1 -- 172.21.15.111:0/2076711187 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc464032440 con 0x7fc474071b40 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.088+0000 7fc478bdb640 1 -- 172.21.15.111:0/2076711187 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464032810 con 0x7fc474071b40 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 -- 172.21.15.111:0/2076711187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474071b40 msgr2=0x7fc47406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 --2- 172.21.15.111:0/2076711187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474071b40 0x7fc47406bc20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc4640097a0 tx=0x7fc46402a180 comp rx=0 tx=0).stop 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 -- 172.21.15.111:0/2076711187 shutdown_connections 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 --2- 172.21.15.111:0/2076711187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474071b40 0x7fc47406bc20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 --2- 172.21.15.111:0/2076711187 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474151740 0x7fc474071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 -- 172.21.15.111:0/2076711187 >> 172.21.15.111:0/2076711187 conn(0x7fc4740b1c00 msgr2=0x7fc4740b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:39.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 -- 172.21.15.111:0/2076711187 shutdown_connections 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.089+0000 7fc47be41640 1 -- 172.21.15.111:0/2076711187 wait complete. 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.090+0000 7fc47be41640 1 Processor -- start 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.090+0000 7fc47be41640 1 -- start start 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47be41640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc47406c160 0x7fc47414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47be41640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474071b40 0x7fc4741f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47be41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 0x7fc47414ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47be41640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc47415af10 con 0x7fc474151740 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47be41640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc474151b20 con 0x7fc47406c160 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47be41640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc47415b090 con 0x7fc474071b40 2024-05-27T00:39:39.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 0x7fc47414ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 0x7fc47414ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56848/0 (socket says 172.21.15.111:56848) 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 -- 172.21.15.111:0/2678999510 learned_addr learned my addr 172.21.15.111:0/2678999510 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 -- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474071b40 msgr2=0x7fc4741f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474071b40 0x7fc4741f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 -- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc47406c160 msgr2=0x7fc47414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc47406c160 0x7fc47414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 -- 172.21.15.111:0/2678999510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47414d7b0 con 0x7fc474151740 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.091+0000 7fc47a3de640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 0x7fc47414ce00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00c420 tx=0x7fc46c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.092+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c017070 con 0x7fc474151740 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.092+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46c005ce0 con 0x7fc474151740 2024-05-27T00:39:39.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.092+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0072d0 con 0x7fc474151740 2024-05-27T00:39:39.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.092+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4740cde40 con 0x7fc474151740 2024-05-27T00:39:39.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.092+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47414da50 con 0x7fc474151740 2024-05-27T00:39:39.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.093+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4741f4830 con 0x7fc474151740 2024-05-27T00:39:39.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.093+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc46c01a070 con 0x7fc474151740 2024-05-27T00:39:39.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.094+0000 7fc46a7fc640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be30 0x7fc45806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.094+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc46c08a3d0 con 0x7fc474151740 2024-05-27T00:39:39.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.094+0000 7fc479bdd640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be30 0x7fc45806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.094+0000 7fc479bdd640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be30 0x7fc45806e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc4700070b0 tx=0x7fc470007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.098+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc46c0583f0 con 0x7fc474151740 2024-05-27T00:39:39.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.272+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc474068680 con 0x7fc474151740 2024-05-27T00:39:39.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.274+0000 7fc46a7fc640 1 -- 172.21.15.111:0/2678999510 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fc46c05c6f0 con 0x7fc474151740 2024-05-27T00:39:39.275 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:39.275 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be30 msgr2=0x7fc45806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be30 0x7fc45806e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc4700070b0 tx=0x7fc470007040 comp rx=0 tx=0).stop 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 msgr2=0x7fc47414ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 0x7fc47414ce00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00c420 tx=0x7fc46c00c8e0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 shutdown_connections 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be30 0x7fc45806e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc474151740 0x7fc47414ce00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc474071b40 0x7fc4741f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 --2- 172.21.15.111:0/2678999510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc47406c160 0x7fc47414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 >> 172.21.15.111:0/2678999510 conn(0x7fc4740b1c00 msgr2=0x7fc47415a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:39.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 shutdown_connections 2024-05-27T00:39:39.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.276+0000 7fc47be41640 1 -- 172.21.15.111:0/2678999510 wait complete. 2024-05-27T00:39:39.288 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-27T00:39:39.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:39:39.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.388+0000 7fd2f51c1640 1 Processor -- start 2024-05-27T00:39:39.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.388+0000 7fd2f51c1640 1 -- start start 2024-05-27T00:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2f51c1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f015f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2f51c1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd2f0162430 0x7fd2f00af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2f51c1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2f00afc30 con 0x7fd2f00af7a0 2024-05-27T00:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2f51c1640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd2f00b4630 con 0x7fd2f0162430 2024-05-27T00:39:39.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2f51c1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f00b3a20 con 0x7fd2f0162050 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2ef59e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd2f00af7a0 0x7fd2f00b4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35444/0 (socket says 172.21.15.111:35444) 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2ef59e640 1 -- 172.21.15.111:0/3691289583 learned_addr learned my addr 172.21.15.111:0/3691289583 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2eed9d640 1 --2- 172.21.15.111:0/3691289583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f015f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2ee59c640 1 --2- 172.21.15.111:0/3691289583 >> v2:172.21.15.153:3300/0 conn(0x7fd2f0162430 0x7fd2f00af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2edd9b640 1 -- 172.21.15.111:0/3691289583 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410444385 0 0) 0x7fd2f00afc30 con 0x7fd2f00af7a0 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2edd9b640 1 -- 172.21.15.111:0/3691289583 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2d8003520 con 0x7fd2f00af7a0 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2eed9d640 1 -- 172.21.15.111:0/3691289583 >> v2:172.21.15.153:3300/0 conn(0x7fd2f0162430 msgr2=0x7fd2f00af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2eed9d640 1 --2- 172.21.15.111:0/3691289583 >> v2:172.21.15.153:3300/0 conn(0x7fd2f0162430 0x7fd2f00af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2eed9d640 1 -- 172.21.15.111:0/3691289583 >> v1:172.21.15.111:6789/0 conn(0x7fd2f00af7a0 legacy=0x7fd2f00b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2eed9d640 1 -- 172.21.15.111:0/3691289583 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f00b3d00 con 0x7fd2f0162050 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.389+0000 7fd2ee59c640 1 --2- 172.21.15.111:0/3691289583 >> v2:172.21.15.153:3300/0 conn(0x7fd2f0162430 0x7fd2f00af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.390+0000 7fd2eed9d640 1 --2- 172.21.15.111:0/3691289583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f015f9b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009d90 tx=0x7fd2e802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ff59f91edc2a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.390+0000 7fd2edd9b640 1 -- 172.21.15.111:0/3691289583 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8007740 con 0x7fd2f0162050 2024-05-27T00:39:39.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.390+0000 7fd2edd9b640 1 -- 172.21.15.111:0/3691289583 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e80078e0 con 0x7fd2f0162050 2024-05-27T00:39:39.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.390+0000 7fd2edd9b640 1 -- 172.21.15.111:0/3691289583 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8007c90 con 0x7fd2f0162050 2024-05-27T00:39:39.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 -- 172.21.15.111:0/3691289583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 msgr2=0x7fd2f015f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/3691289583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f015f9b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009d90 tx=0x7fd2e802a660 comp rx=0 tx=0).stop 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 -- 172.21.15.111:0/3691289583 shutdown_connections 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/3691289583 >> v2:172.21.15.153:3300/0 conn(0x7fd2f0162430 0x7fd2f00af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/3691289583 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f015f9b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 -- 172.21.15.111:0/3691289583 >> 172.21.15.111:0/3691289583 conn(0x7fd2f0072a30 msgr2=0x7fd2f006bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 -- 172.21.15.111:0/3691289583 shutdown_connections 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.391+0000 7fd2f51c1640 1 -- 172.21.15.111:0/3691289583 wait complete. 2024-05-27T00:39:39.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.392+0000 7fd2f51c1640 1 Processor -- start 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.392+0000 7fd2f51c1640 1 -- start start 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2f51c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 0x7fd2f01461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2f51c1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f00cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2f51c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f0162430 0x7fd2f00c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2f51c1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f00b51d0 con 0x7fd2f00af7a0 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2f51c1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f00b42c0 con 0x7fd2f0162430 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2f51c1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f00b5350 con 0x7fd2f0162050 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2eed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 0x7fd2f01461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2eed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 0x7fd2f01461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56850/0 (socket says 172.21.15.111:56850) 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2eed9d640 1 -- 172.21.15.111:0/834614347 learned_addr learned my addr 172.21.15.111:0/834614347 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2ef59e640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f0162430 0x7fd2f00c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2eed9d640 1 -- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 msgr2=0x7fd2f00cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.393+0000 7fd2ee59c640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f00cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2eed9d640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f00cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2eed9d640 1 -- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f0162430 msgr2=0x7fd2f00c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2eed9d640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f0162430 0x7fd2f00c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2eed9d640 1 -- 172.21.15.111:0/834614347 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f00cf740 con 0x7fd2f00af7a0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2eed9d640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 0x7fd2f01461a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2e802ac50 tx=0x7fd2e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8040070 con 0x7fd2f00af7a0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8031c40 con 0x7fd2f00af7a0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f00ce6a0 con 0x7fd2f00af7a0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8009450 con 0x7fd2f00af7a0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.394+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f00cec60 con 0x7fd2f00af7a0 2024-05-27T00:39:39.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.395+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f0068680 con 0x7fd2f00af7a0 2024-05-27T00:39:39.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.395+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2e8039040 con 0x7fd2f00af7a0 2024-05-27T00:39:39.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.396+0000 7fd2d77fe640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd2dc06be60 0x7fd2dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.396+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd2e80aca30 con 0x7fd2f00af7a0 2024-05-27T00:39:39.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.396+0000 7fd2ee59c640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd2dc06be60 0x7fd2dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.397+0000 7fd2ee59c640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd2dc06be60 0x7fd2dc06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00cc5b0 tx=0x7fd2e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.400+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd2e807b0a0 con 0x7fd2f00af7a0 2024-05-27T00:39:39.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.570+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2f00b0ce0 con 0x7fd2dc06be60 2024-05-27T00:39:39.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.579+0000 7fd2d77fe640 1 -- 172.21.15.111:0/834614347 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fd2f00b0ce0 con 0x7fd2dc06be60 2024-05-27T00:39:39.584 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:39:39.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.582+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd2dc06be60 msgr2=0x7fd2dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.582+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd2dc06be60 0x7fd2dc06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00cc5b0 tx=0x7fd2e4008040 comp rx=0 tx=0).stop 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.582+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 msgr2=0x7fd2f01461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.582+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 0x7fd2f01461a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2e802ac50 tx=0x7fd2e8004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 shutdown_connections 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd2dc06be60 0x7fd2dc06e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2f0162430 0x7fd2f00c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd2f0162050 0x7fd2f00cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 --2- 172.21.15.111:0/834614347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2f00af7a0 0x7fd2f01461a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 >> 172.21.15.111:0/834614347 conn(0x7fd2f0072a30 msgr2=0x7fd2f006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 shutdown_connections 2024-05-27T00:39:39.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.583+0000 7fd2f51c1640 1 -- 172.21.15.111:0/834614347 wait complete. 2024-05-27T00:39:39.594 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-27T00:39:38.928518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2048876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.320081"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133844+0000","last_change":"2024-05-27T00:39:20.574932+0000","last_active":"2024-05-27T00:39:28.133844+0000","last_peered":"2024-05-27T00:39:28.133844+0000","last_clean":"2024-05-27T00:39:28.133844+0000","last_became_active":"2024-05-27T00:39:20.574617+0000","last_became_peered":"2024-05-27T00:39:20.574617+0000","last_unstale":"2024-05-27T00:39:28.133844+0000","last_undegraded":"2024-05-27T00:39:28.133844+0000","last_fullsized":"2024-05-27T00:39:28.133844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610219+0000","last_change":"2024-05-27T00:39:21.594550+0000","last_active":"2024-05-27T00:39:27.610219+0000","last_peered":"2024-05-27T00:39:27.610219+0000","last_clean":"2024-05-27T00:39:27.610219+0000","last_became_active":"2024-05-27T00:39:21.594296+0000","last_became_peered":"2024-05-27T00:39:21.594296+0000","last_unstale":"2024-05-27T00:39:27.610219+0000","last_undegraded":"2024-05-27T00:39:27.610219+0000","last_fullsized":"2024-05-27T00:39:27.610219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609737+0000","last_change":"2024-05-27T00:39:25.965194+0000","last_active":"2024-05-27T00:39:27.609737+0000","last_peered":"2024-05-27T00:39:27.609737+0000","last_clean":"2024-05-27T00:39:27.609737+0000","last_became_active":"2024-05-27T00:39:25.964829+0000","last_became_peered":"2024-05-27T00:39:25.964829+0000","last_unstale":"2024-05-27T00:39:27.609737+0000","last_undegraded":"2024-05-27T00:39:27.609737+0000","last_fullsized":"2024-05-27T00:39:27.609737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:38.727603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610308+0000","last_change":"2024-05-27T00:39:20.569739+0000","last_active":"2024-05-27T00:39:27.610308+0000","last_peered":"2024-05-27T00:39:27.610308+0000","last_clean":"2024-05-27T00:39:27.610308+0000","last_became_active":"2024-05-27T00:39:20.569481+0000","last_became_peered":"2024-05-27T00:39:20.569481+0000","last_unstale":"2024-05-27T00:39:27.610308+0000","last_undegraded":"2024-05-27T00:39:27.610308+0000","last_fullsized":"2024-05-27T00:39:27.610308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133753+0000","last_change":"2024-05-27T00:39:21.576759+0000","last_active":"2024-05-27T00:39:28.133753+0000","last_peered":"2024-05-27T00:39:28.133753+0000","last_clean":"2024-05-27T00:39:28.133753+0000","last_became_active":"2024-05-27T00:39:21.576492+0000","last_became_peered":"2024-05-27T00:39:21.576492+0000","last_unstale":"2024-05-27T00:39:28.133753+0000","last_undegraded":"2024-05-27T00:39:28.133753+0000","last_fullsized":"2024-05-27T00:39:28.133753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833532+0000","last_change":"2024-05-27T00:39:25.626523+0000","last_active":"2024-05-27T00:39:28.833532+0000","last_peered":"2024-05-27T00:39:28.833532+0000","last_clean":"2024-05-27T00:39:28.833532+0000","last_became_active":"2024-05-27T00:39:25.626132+0000","last_became_peered":"2024-05-27T00:39:25.626132+0000","last_unstale":"2024-05-27T00:39:28.833532+0000","last_undegraded":"2024-05-27T00:39:28.833532+0000","last_fullsized":"2024-05-27T00:39:28.833532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834538+0000","last_change":"2024-05-27T00:39:20.577889+0000","last_active":"2024-05-27T00:39:28.834538+0000","last_peered":"2024-05-27T00:39:28.834538+0000","last_clean":"2024-05-27T00:39:28.834538+0000","last_became_active":"2024-05-27T00:39:20.577565+0000","last_became_peered":"2024-05-27T00:39:20.577565+0000","last_unstale":"2024-05-27T00:39:28.834538+0000","last_undegraded":"2024-05-27T00:39:28.834538+0000","last_fullsized":"2024-05-27T00:39:28.834538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:12:22.836110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610522+0000","last_change":"2024-05-27T00:39:21.580044+0000","last_active":"2024-05-27T00:39:27.610522+0000","last_peered":"2024-05-27T00:39:27.610522+0000","last_clean":"2024-05-27T00:39:27.610522+0000","last_became_active":"2024-05-27T00:39:21.579770+0000","last_became_peered":"2024-05-27T00:39:21.579770+0000","last_unstale":"2024-05-27T00:39:27.610522+0000","last_undegraded":"2024-05-27T00:39:27.610522+0000","last_fullsized":"2024-05-27T00:39:27.610522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833971+0000","last_change":"2024-05-27T00:39:25.627365+0000","last_active":"2024-05-27T00:39:28.833971+0000","last_peered":"2024-05-27T00:39:28.833971+0000","last_clean":"2024-05-27T00:39:28.833971+0000","last_became_active":"2024-05-27T00:39:25.626973+0000","last_became_peered":"2024-05-27T00:39:25.626973+0000","last_unstale":"2024-05-27T00:39:28.833971+0000","last_undegraded":"2024-05-27T00:39:28.833971+0000","last_fullsized":"2024-05-27T00:39:28.833971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833143+0000","last_change":"2024-05-27T00:39:20.577815+0000","last_active":"2024-05-27T00:39:28.833143+0000","last_peered":"2024-05-27T00:39:28.833143+0000","last_clean":"2024-05-27T00:39:28.833143+0000","last_became_active":"2024-05-27T00:39:20.577495+0000","last_became_peered":"2024-05-27T00:39:20.577495+0000","last_unstale":"2024-05-27T00:39:28.833143+0000","last_undegraded":"2024-05-27T00:39:28.833143+0000","last_fullsized":"2024-05-27T00:39:28.833143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:05.446553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610465+0000","last_change":"2024-05-27T00:39:21.576226+0000","last_active":"2024-05-27T00:39:27.610465+0000","last_peered":"2024-05-27T00:39:27.610465+0000","last_clean":"2024-05-27T00:39:27.610465+0000","last_became_active":"2024-05-27T00:39:21.575940+0000","last_became_peered":"2024-05-27T00:39:21.575940+0000","last_unstale":"2024-05-27T00:39:27.610465+0000","last_undegraded":"2024-05-27T00:39:27.610465+0000","last_fullsized":"2024-05-27T00:39:27.610465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833014+0000","last_change":"2024-05-27T00:39:25.623493+0000","last_active":"2024-05-27T00:39:28.833014+0000","last_peered":"2024-05-27T00:39:28.833014+0000","last_clean":"2024-05-27T00:39:28.833014+0000","last_became_active":"2024-05-27T00:39:25.623101+0000","last_became_peered":"2024-05-27T00:39:25.623101+0000","last_unstale":"2024-05-27T00:39:28.833014+0000","last_undegraded":"2024-05-27T00:39:28.833014+0000","last_fullsized":"2024-05-27T00:39:28.833014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:12.831089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834471+0000","last_change":"2024-05-27T00:39:21.594839+0000","last_active":"2024-05-27T00:39:28.834471+0000","last_peered":"2024-05-27T00:39:28.834471+0000","last_clean":"2024-05-27T00:39:28.834471+0000","last_became_active":"2024-05-27T00:39:21.594428+0000","last_became_peered":"2024-05-27T00:39:21.594428+0000","last_unstale":"2024-05-27T00:39:28.834471+0000","last_undegraded":"2024-05-27T00:39:28.834471+0000","last_fullsized":"2024-05-27T00:39:28.834471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833832+0000","last_change":"2024-05-27T00:39:20.579815+0000","last_active":"2024-05-27T00:39:28.833832+0000","last_peered":"2024-05-27T00:39:28.833832+0000","last_clean":"2024-05-27T00:39:28.833832+0000","last_became_active":"2024-05-27T00:39:20.579507+0000","last_became_peered":"2024-05-27T00:39:20.579507+0000","last_unstale":"2024-05-27T00:39:28.833832+0000","last_undegraded":"2024-05-27T00:39:28.833832+0000","last_fullsized":"2024-05-27T00:39:28.833832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:17.368730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833761+0000","last_change":"2024-05-27T00:39:25.625823+0000","last_active":"2024-05-27T00:39:28.833761+0000","last_peered":"2024-05-27T00:39:28.833761+0000","last_clean":"2024-05-27T00:39:28.833761+0000","last_became_active":"2024-05-27T00:39:25.625378+0000","last_became_peered":"2024-05-27T00:39:25.625378+0000","last_unstale":"2024-05-27T00:39:28.833761+0000","last_undegraded":"2024-05-27T00:39:28.833761+0000","last_fullsized":"2024-05-27T00:39:28.833761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609891+0000","last_change":"2024-05-27T00:39:21.591774+0000","last_active":"2024-05-27T00:39:27.609891+0000","last_peered":"2024-05-27T00:39:27.609891+0000","last_clean":"2024-05-27T00:39:27.609891+0000","last_became_active":"2024-05-27T00:39:21.591413+0000","last_became_peered":"2024-05-27T00:39:21.591413+0000","last_unstale":"2024-05-27T00:39:27.609891+0000","last_undegraded":"2024-05-27T00:39:27.609891+0000","last_fullsized":"2024-05-27T00:39:27.609891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604467+0000","last_change":"2024-05-27T00:39:20.562227+0000","last_active":"2024-05-27T00:39:27.604467+0000","last_peered":"2024-05-27T00:39:27.604467+0000","last_clean":"2024-05-27T00:39:27.604467+0000","last_became_active":"2024-05-27T00:39:20.561938+0000","last_became_peered":"2024-05-27T00:39:20.561938+0000","last_unstale":"2024-05-27T00:39:27.604467+0000","last_undegraded":"2024-05-27T00:39:27.604467+0000","last_fullsized":"2024-05-27T00:39:27.604467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834385+0000","last_change":"2024-05-27T00:39:25.628730+0000","last_active":"2024-05-27T00:39:28.834385+0000","last_peered":"2024-05-27T00:39:28.834385+0000","last_clean":"2024-05-27T00:39:28.834385+0000","last_became_active":"2024-05-27T00:39:25.628352+0000","last_became_peered":"2024-05-27T00:39:25.628352+0000","last_unstale":"2024-05-27T00:39:28.834385+0000","last_undegraded":"2024-05-27T00:39:28.834385+0000","last_fullsized":"2024-05-27T00:39:28.834385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609789+0000","last_change":"2024-05-27T00:39:21.585438+0000","last_active":"2024-05-27T00:39:27.609789+0000","last_peered":"2024-05-27T00:39:27.609789+0000","last_clean":"2024-05-27T00:39:27.609789+0000","last_became_active":"2024-05-27T00:39:21.585062+0000","last_became_peered":"2024-05-27T00:39:21.585062+0000","last_unstale":"2024-05-27T00:39:27.609789+0000","last_undegraded":"2024-05-27T00:39:27.609789+0000","last_fullsized":"2024-05-27T00:39:27.609789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604165+0000","last_change":"2024-05-27T00:39:20.570857+0000","last_active":"2024-05-27T00:39:27.604165+0000","last_peered":"2024-05-27T00:39:27.604165+0000","last_clean":"2024-05-27T00:39:27.604165+0000","last_became_active":"2024-05-27T00:39:20.570564+0000","last_became_peered":"2024-05-27T00:39:20.570564+0000","last_unstale":"2024-05-27T00:39:27.604165+0000","last_undegraded":"2024-05-27T00:39:27.604165+0000","last_fullsized":"2024-05-27T00:39:27.604165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605057+0000","last_change":"2024-05-27T00:39:25.963988+0000","last_active":"2024-05-27T00:39:27.605057+0000","last_peered":"2024-05-27T00:39:27.605057+0000","last_clean":"2024-05-27T00:39:27.605057+0000","last_became_active":"2024-05-27T00:39:25.963532+0000","last_became_peered":"2024-05-27T00:39:25.963532+0000","last_unstale":"2024-05-27T00:39:27.605057+0000","last_undegraded":"2024-05-27T00:39:27.605057+0000","last_fullsized":"2024-05-27T00:39:27.605057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833654+0000","last_change":"2024-05-27T00:39:21.590869+0000","last_active":"2024-05-27T00:39:28.833654+0000","last_peered":"2024-05-27T00:39:28.833654+0000","last_clean":"2024-05-27T00:39:28.833654+0000","last_became_active":"2024-05-27T00:39:21.584136+0000","last_became_peered":"2024-05-27T00:39:21.584136+0000","last_unstale":"2024-05-27T00:39:28.833654+0000","last_undegraded":"2024-05-27T00:39:28.833654+0000","last_fullsized":"2024-05-27T00:39:28.833654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:48.264046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832474+0000","last_change":"2024-05-27T00:39:20.579097+0000","last_active":"2024-05-27T00:39:28.832474+0000","last_peered":"2024-05-27T00:39:28.832474+0000","last_clean":"2024-05-27T00:39:28.832474+0000","last_became_active":"2024-05-27T00:39:20.578796+0000","last_became_peered":"2024-05-27T00:39:20.578796+0000","last_unstale":"2024-05-27T00:39:28.832474+0000","last_undegraded":"2024-05-27T00:39:28.832474+0000","last_fullsized":"2024-05-27T00:39:28.832474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833500+0000","last_change":"2024-05-27T00:39:25.628322+0000","last_active":"2024-05-27T00:39:28.833500+0000","last_peered":"2024-05-27T00:39:28.833500+0000","last_clean":"2024-05-27T00:39:28.833500+0000","last_became_active":"2024-05-27T00:39:25.627878+0000","last_became_peered":"2024-05-27T00:39:25.627878+0000","last_unstale":"2024-05-27T00:39:28.833500+0000","last_undegraded":"2024-05-27T00:39:28.833500+0000","last_fullsized":"2024-05-27T00:39:28.833500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:42.049691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133677+0000","last_change":"2024-05-27T00:39:21.581752+0000","last_active":"2024-05-27T00:39:28.133677+0000","last_peered":"2024-05-27T00:39:28.133677+0000","last_clean":"2024-05-27T00:39:28.133677+0000","last_became_active":"2024-05-27T00:39:21.581488+0000","last_became_peered":"2024-05-27T00:39:21.581488+0000","last_unstale":"2024-05-27T00:39:28.133677+0000","last_undegraded":"2024-05-27T00:39:28.133677+0000","last_fullsized":"2024-05-27T00:39:28.133677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835187+0000","last_change":"2024-05-27T00:39:20.578119+0000","last_active":"2024-05-27T00:39:28.835187+0000","last_peered":"2024-05-27T00:39:28.835187+0000","last_clean":"2024-05-27T00:39:28.835187+0000","last_became_active":"2024-05-27T00:39:20.577830+0000","last_became_peered":"2024-05-27T00:39:20.577830+0000","last_unstale":"2024-05-27T00:39:28.835187+0000","last_undegraded":"2024-05-27T00:39:28.835187+0000","last_fullsized":"2024-05-27T00:39:28.835187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832957+0000","last_change":"2024-05-27T00:39:25.965993+0000","last_active":"2024-05-27T00:39:28.832957+0000","last_peered":"2024-05-27T00:39:28.832957+0000","last_clean":"2024-05-27T00:39:28.832957+0000","last_became_active":"2024-05-27T00:39:25.965384+0000","last_became_peered":"2024-05-27T00:39:25.965384+0000","last_unstale":"2024-05-27T00:39:28.832957+0000","last_undegraded":"2024-05-27T00:39:28.832957+0000","last_fullsized":"2024-05-27T00:39:28.832957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610354+0000","last_change":"2024-05-27T00:39:21.591396+0000","last_active":"2024-05-27T00:39:27.610354+0000","last_peered":"2024-05-27T00:39:27.610354+0000","last_clean":"2024-05-27T00:39:27.610354+0000","last_became_active":"2024-05-27T00:39:21.591116+0000","last_became_peered":"2024-05-27T00:39:21.591116+0000","last_unstale":"2024-05-27T00:39:27.610354+0000","last_undegraded":"2024-05-27T00:39:27.610354+0000","last_fullsized":"2024-05-27T00:39:27.610354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604797+0000","last_change":"2024-05-27T00:39:20.573838+0000","last_active":"2024-05-27T00:39:27.604797+0000","last_peered":"2024-05-27T00:39:27.604797+0000","last_clean":"2024-05-27T00:39:27.604797+0000","last_became_active":"2024-05-27T00:39:20.573515+0000","last_became_peered":"2024-05-27T00:39:20.573515+0000","last_unstale":"2024-05-27T00:39:27.604797+0000","last_undegraded":"2024-05-27T00:39:27.604797+0000","last_fullsized":"2024-05-27T00:39:27.604797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609614+0000","last_change":"2024-05-27T00:39:25.960920+0000","last_active":"2024-05-27T00:39:27.609614+0000","last_peered":"2024-05-27T00:39:27.609614+0000","last_clean":"2024-05-27T00:39:27.609614+0000","last_became_active":"2024-05-27T00:39:25.960487+0000","last_became_peered":"2024-05-27T00:39:25.960487+0000","last_unstale":"2024-05-27T00:39:27.609614+0000","last_undegraded":"2024-05-27T00:39:27.609614+0000","last_fullsized":"2024-05-27T00:39:27.609614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:59.647950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604641+0000","last_change":"2024-05-27T00:39:21.583980+0000","last_active":"2024-05-27T00:39:27.604641+0000","last_peered":"2024-05-27T00:39:27.604641+0000","last_clean":"2024-05-27T00:39:27.604641+0000","last_became_active":"2024-05-27T00:39:21.583700+0000","last_became_peered":"2024-05-27T00:39:21.583700+0000","last_unstale":"2024-05-27T00:39:27.604641+0000","last_undegraded":"2024-05-27T00:39:27.604641+0000","last_fullsized":"2024-05-27T00:39:27.604641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834010+0000","last_change":"2024-05-27T00:39:20.579198+0000","last_active":"2024-05-27T00:39:28.834010+0000","last_peered":"2024-05-27T00:39:28.834010+0000","last_clean":"2024-05-27T00:39:28.834010+0000","last_became_active":"2024-05-27T00:39:20.578902+0000","last_became_peered":"2024-05-27T00:39:20.578902+0000","last_unstale":"2024-05-27T00:39:28.834010+0000","last_undegraded":"2024-05-27T00:39:28.834010+0000","last_fullsized":"2024-05-27T00:39:28.834010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.610865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.962036+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.961152+0000","last_became_peered":"2024-05-27T00:39:25.961152+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610557+0000","last_change":"2024-05-27T00:39:21.591906+0000","last_active":"2024-05-27T00:39:27.610557+0000","last_peered":"2024-05-27T00:39:27.610557+0000","last_clean":"2024-05-27T00:39:27.610557+0000","last_became_active":"2024-05-27T00:39:21.591611+0000","last_became_peered":"2024-05-27T00:39:21.591611+0000","last_unstale":"2024-05-27T00:39:27.610557+0000","last_undegraded":"2024-05-27T00:39:27.610557+0000","last_fullsized":"2024-05-27T00:39:27.610557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604735+0000","last_change":"2024-05-27T00:39:20.569834+0000","last_active":"2024-05-27T00:39:27.604735+0000","last_peered":"2024-05-27T00:39:27.604735+0000","last_clean":"2024-05-27T00:39:27.604735+0000","last_became_active":"2024-05-27T00:39:20.569540+0000","last_became_peered":"2024-05-27T00:39:20.569540+0000","last_unstale":"2024-05-27T00:39:27.604735+0000","last_undegraded":"2024-05-27T00:39:27.604735+0000","last_fullsized":"2024-05-27T00:39:27.604735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604262+0000","last_change":"2024-05-27T00:39:25.627533+0000","last_active":"2024-05-27T00:39:27.604262+0000","last_peered":"2024-05-27T00:39:27.604262+0000","last_clean":"2024-05-27T00:39:27.604262+0000","last_became_active":"2024-05-27T00:39:25.625872+0000","last_became_peered":"2024-05-27T00:39:25.625872+0000","last_unstale":"2024-05-27T00:39:27.604262+0000","last_undegraded":"2024-05-27T00:39:27.604262+0000","last_fullsized":"2024-05-27T00:39:27.604262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834370+0000","last_change":"2024-05-27T00:39:21.598984+0000","last_active":"2024-05-27T00:39:28.834370+0000","last_peered":"2024-05-27T00:39:28.834370+0000","last_clean":"2024-05-27T00:39:28.834370+0000","last_became_active":"2024-05-27T00:39:21.594222+0000","last_became_peered":"2024-05-27T00:39:21.594222+0000","last_unstale":"2024-05-27T00:39:28.834370+0000","last_undegraded":"2024-05-27T00:39:28.834370+0000","last_fullsized":"2024-05-27T00:39:28.834370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610187+0000","last_change":"2024-05-27T00:39:20.565350+0000","last_active":"2024-05-27T00:39:27.610187+0000","last_peered":"2024-05-27T00:39:27.610187+0000","last_clean":"2024-05-27T00:39:27.610187+0000","last_became_active":"2024-05-27T00:39:20.562112+0000","last_became_peered":"2024-05-27T00:39:20.562112+0000","last_unstale":"2024-05-27T00:39:27.610187+0000","last_undegraded":"2024-05-27T00:39:27.610187+0000","last_fullsized":"2024-05-27T00:39:27.610187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604681+0000","last_change":"2024-05-27T00:39:25.624845+0000","last_active":"2024-05-27T00:39:27.604681+0000","last_peered":"2024-05-27T00:39:27.604681+0000","last_clean":"2024-05-27T00:39:27.604681+0000","last_became_active":"2024-05-27T00:39:25.624336+0000","last_became_peered":"2024-05-27T00:39:25.624336+0000","last_unstale":"2024-05-27T00:39:27.604681+0000","last_undegraded":"2024-05-27T00:39:27.604681+0000","last_fullsized":"2024-05-27T00:39:27.604681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604350+0000","last_change":"2024-05-27T00:39:21.586786+0000","last_active":"2024-05-27T00:39:27.604350+0000","last_peered":"2024-05-27T00:39:27.604350+0000","last_clean":"2024-05-27T00:39:27.604350+0000","last_became_active":"2024-05-27T00:39:21.586521+0000","last_became_peered":"2024-05-27T00:39:21.586521+0000","last_unstale":"2024-05-27T00:39:27.604350+0000","last_undegraded":"2024-05-27T00:39:27.604350+0000","last_fullsized":"2024-05-27T00:39:27.604350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573104+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.572665+0000","last_became_peered":"2024-05-27T00:39:20.572665+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605183+0000","last_change":"2024-05-27T00:39:25.625105+0000","last_active":"2024-05-27T00:39:27.605183+0000","last_peered":"2024-05-27T00:39:27.605183+0000","last_clean":"2024-05-27T00:39:27.605183+0000","last_became_active":"2024-05-27T00:39:25.624400+0000","last_became_peered":"2024-05-27T00:39:25.624400+0000","last_unstale":"2024-05-27T00:39:27.605183+0000","last_undegraded":"2024-05-27T00:39:27.605183+0000","last_fullsized":"2024-05-27T00:39:27.605183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133189+0000","last_change":"2024-05-27T00:39:21.578555+0000","last_active":"2024-05-27T00:39:28.133189+0000","last_peered":"2024-05-27T00:39:28.133189+0000","last_clean":"2024-05-27T00:39:28.133189+0000","last_became_active":"2024-05-27T00:39:21.578291+0000","last_became_peered":"2024-05-27T00:39:21.578291+0000","last_unstale":"2024-05-27T00:39:28.133189+0000","last_undegraded":"2024-05-27T00:39:28.133189+0000","last_fullsized":"2024-05-27T00:39:28.133189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:00.616291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832328+0000","last_change":"2024-05-27T00:39:20.578837+0000","last_active":"2024-05-27T00:39:28.832328+0000","last_peered":"2024-05-27T00:39:28.832328+0000","last_clean":"2024-05-27T00:39:28.832328+0000","last_became_active":"2024-05-27T00:39:20.578414+0000","last_became_peered":"2024-05-27T00:39:20.578414+0000","last_unstale":"2024-05-27T00:39:28.832328+0000","last_undegraded":"2024-05-27T00:39:28.832328+0000","last_fullsized":"2024-05-27T00:39:28.832328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133680+0000","last_change":"2024-05-27T00:39:25.962038+0000","last_active":"2024-05-27T00:39:28.133680+0000","last_peered":"2024-05-27T00:39:28.133680+0000","last_clean":"2024-05-27T00:39:28.133680+0000","last_became_active":"2024-05-27T00:39:25.961489+0000","last_became_peered":"2024-05-27T00:39:25.961489+0000","last_unstale":"2024-05-27T00:39:28.133680+0000","last_undegraded":"2024-05-27T00:39:28.133680+0000","last_fullsized":"2024-05-27T00:39:28.133680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:36.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609779+0000","last_change":"2024-05-27T00:39:21.593420+0000","last_active":"2024-05-27T00:39:27.609779+0000","last_peered":"2024-05-27T00:39:27.609779+0000","last_clean":"2024-05-27T00:39:27.609779+0000","last_became_active":"2024-05-27T00:39:21.593165+0000","last_became_peered":"2024-05-27T00:39:21.593165+0000","last_unstale":"2024-05-27T00:39:27.609779+0000","last_undegraded":"2024-05-27T00:39:27.609779+0000","last_fullsized":"2024-05-27T00:39:27.609779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610340+0000","last_change":"2024-05-27T00:39:20.573661+0000","last_active":"2024-05-27T00:39:27.610340+0000","last_peered":"2024-05-27T00:39:27.610340+0000","last_clean":"2024-05-27T00:39:27.610340+0000","last_became_active":"2024-05-27T00:39:20.572675+0000","last_became_peered":"2024-05-27T00:39:20.572675+0000","last_unstale":"2024-05-27T00:39:27.610340+0000","last_undegraded":"2024-05-27T00:39:27.610340+0000","last_fullsized":"2024-05-27T00:39:27.610340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832965+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:39:28.832965+0000","last_peered":"2024-05-27T00:39:28.832965+0000","last_clean":"2024-05-27T00:39:28.832965+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:39:28.832965+0000","last_undegraded":"2024-05-27T00:39:28.832965+0000","last_fullsized":"2024-05-27T00:39:28.832965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834052+0000","last_change":"2024-05-27T00:39:25.964404+0000","last_active":"2024-05-27T00:39:28.834052+0000","last_peered":"2024-05-27T00:39:28.834052+0000","last_clean":"2024-05-27T00:39:28.834052+0000","last_became_active":"2024-05-27T00:39:25.963955+0000","last_became_peered":"2024-05-27T00:39:25.963955+0000","last_unstale":"2024-05-27T00:39:28.834052+0000","last_undegraded":"2024-05-27T00:39:28.834052+0000","last_fullsized":"2024-05-27T00:39:28.834052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:48.698768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834325+0000","last_change":"2024-05-27T00:39:21.585811+0000","last_active":"2024-05-27T00:39:28.834325+0000","last_peered":"2024-05-27T00:39:28.834325+0000","last_clean":"2024-05-27T00:39:28.834325+0000","last_became_active":"2024-05-27T00:39:21.585466+0000","last_became_peered":"2024-05-27T00:39:21.585466+0000","last_unstale":"2024-05-27T00:39:28.834325+0000","last_undegraded":"2024-05-27T00:39:28.834325+0000","last_fullsized":"2024-05-27T00:39:28.834325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.582020+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.581718+0000","last_became_peered":"2024-05-27T00:39:20.581718+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834411+0000","last_change":"2024-05-27T00:39:25.961052+0000","last_active":"2024-05-27T00:39:28.834411+0000","last_peered":"2024-05-27T00:39:28.834411+0000","last_clean":"2024-05-27T00:39:28.834411+0000","last_became_active":"2024-05-27T00:39:25.960632+0000","last_became_peered":"2024-05-27T00:39:25.960632+0000","last_unstale":"2024-05-27T00:39:28.834411+0000","last_undegraded":"2024-05-27T00:39:28.834411+0000","last_fullsized":"2024-05-27T00:39:28.834411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610612+0000","last_change":"2024-05-27T00:39:21.587149+0000","last_active":"2024-05-27T00:39:27.610612+0000","last_peered":"2024-05-27T00:39:27.610612+0000","last_clean":"2024-05-27T00:39:27.610612+0000","last_became_active":"2024-05-27T00:39:21.585877+0000","last_became_peered":"2024-05-27T00:39:21.585877+0000","last_unstale":"2024-05-27T00:39:27.610612+0000","last_undegraded":"2024-05-27T00:39:27.610612+0000","last_fullsized":"2024-05-27T00:39:27.610612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604682+0000","last_change":"2024-05-27T00:39:20.575827+0000","last_active":"2024-05-27T00:39:27.604682+0000","last_peered":"2024-05-27T00:39:27.604682+0000","last_clean":"2024-05-27T00:39:27.604682+0000","last_became_active":"2024-05-27T00:39:20.575555+0000","last_became_peered":"2024-05-27T00:39:20.575555+0000","last_unstale":"2024-05-27T00:39:27.604682+0000","last_undegraded":"2024-05-27T00:39:27.604682+0000","last_fullsized":"2024-05-27T00:39:27.604682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133662+0000","last_change":"2024-05-27T00:39:25.626022+0000","last_active":"2024-05-27T00:39:28.133662+0000","last_peered":"2024-05-27T00:39:28.133662+0000","last_clean":"2024-05-27T00:39:28.133662+0000","last_became_active":"2024-05-27T00:39:25.625643+0000","last_became_peered":"2024-05-27T00:39:25.625643+0000","last_unstale":"2024-05-27T00:39:28.133662+0000","last_undegraded":"2024-05-27T00:39:28.133662+0000","last_fullsized":"2024-05-27T00:39:28.133662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833753+0000","last_change":"2024-05-27T00:39:21.599339+0000","last_active":"2024-05-27T00:39:28.833753+0000","last_peered":"2024-05-27T00:39:28.833753+0000","last_clean":"2024-05-27T00:39:28.833753+0000","last_became_active":"2024-05-27T00:39:21.598994+0000","last_became_peered":"2024-05-27T00:39:21.598994+0000","last_unstale":"2024-05-27T00:39:28.833753+0000","last_undegraded":"2024-05-27T00:39:28.833753+0000","last_fullsized":"2024-05-27T00:39:28.833753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:13.702562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831770+0000","last_change":"2024-05-27T00:39:20.572771+0000","last_active":"2024-05-27T00:39:28.831770+0000","last_peered":"2024-05-27T00:39:28.831770+0000","last_clean":"2024-05-27T00:39:28.831770+0000","last_became_active":"2024-05-27T00:39:20.572486+0000","last_became_peered":"2024-05-27T00:39:20.572486+0000","last_unstale":"2024-05-27T00:39:28.831770+0000","last_undegraded":"2024-05-27T00:39:28.831770+0000","last_fullsized":"2024-05-27T00:39:28.831770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609790+0000","last_change":"2024-05-27T00:39:25.962926+0000","last_active":"2024-05-27T00:39:27.609790+0000","last_peered":"2024-05-27T00:39:27.609790+0000","last_clean":"2024-05-27T00:39:27.609790+0000","last_became_active":"2024-05-27T00:39:25.962471+0000","last_became_peered":"2024-05-27T00:39:25.962471+0000","last_unstale":"2024-05-27T00:39:27.609790+0000","last_undegraded":"2024-05-27T00:39:27.609790+0000","last_fullsized":"2024-05-27T00:39:27.609790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833146+0000","last_change":"2024-05-27T00:39:21.596571+0000","last_active":"2024-05-27T00:39:28.833146+0000","last_peered":"2024-05-27T00:39:28.833146+0000","last_clean":"2024-05-27T00:39:28.833146+0000","last_became_active":"2024-05-27T00:39:21.596278+0000","last_became_peered":"2024-05-27T00:39:21.596278+0000","last_unstale":"2024-05-27T00:39:28.833146+0000","last_undegraded":"2024-05-27T00:39:28.833146+0000","last_fullsized":"2024-05-27T00:39:28.833146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:42.881822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832599+0000","last_change":"2024-05-27T00:39:20.575461+0000","last_active":"2024-05-27T00:39:28.832599+0000","last_peered":"2024-05-27T00:39:28.832599+0000","last_clean":"2024-05-27T00:39:28.832599+0000","last_became_active":"2024-05-27T00:39:20.575168+0000","last_became_peered":"2024-05-27T00:39:20.575168+0000","last_unstale":"2024-05-27T00:39:28.832599+0000","last_undegraded":"2024-05-27T00:39:28.832599+0000","last_fullsized":"2024-05-27T00:39:28.832599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:04:51.312690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832581+0000","last_change":"2024-05-27T00:39:25.627556+0000","last_active":"2024-05-27T00:39:28.832581+0000","last_peered":"2024-05-27T00:39:28.832581+0000","last_clean":"2024-05-27T00:39:28.832581+0000","last_became_active":"2024-05-27T00:39:25.627169+0000","last_became_peered":"2024-05-27T00:39:25.627169+0000","last_unstale":"2024-05-27T00:39:28.832581+0000","last_undegraded":"2024-05-27T00:39:28.832581+0000","last_fullsized":"2024-05-27T00:39:28.832581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832699+0000","last_change":"2024-05-27T00:39:21.596905+0000","last_active":"2024-05-27T00:39:28.832699+0000","last_peered":"2024-05-27T00:39:28.832699+0000","last_clean":"2024-05-27T00:39:28.832699+0000","last_became_active":"2024-05-27T00:39:21.596586+0000","last_became_peered":"2024-05-27T00:39:21.596586+0000","last_unstale":"2024-05-27T00:39:28.832699+0000","last_undegraded":"2024-05-27T00:39:28.832699+0000","last_fullsized":"2024-05-27T00:39:28.832699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573588+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.573302+0000","last_became_peered":"2024-05-27T00:39:20.573302+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.620609+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.620185+0000","last_became_peered":"2024-05-27T00:39:25.620185+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835033+0000","last_change":"2024-05-27T00:39:21.593089+0000","last_active":"2024-05-27T00:39:28.835033+0000","last_peered":"2024-05-27T00:39:28.835033+0000","last_clean":"2024-05-27T00:39:28.835033+0000","last_became_active":"2024-05-27T00:39:21.592034+0000","last_became_peered":"2024-05-27T00:39:21.592034+0000","last_unstale":"2024-05-27T00:39:28.835033+0000","last_undegraded":"2024-05-27T00:39:28.835033+0000","last_fullsized":"2024-05-27T00:39:28.835033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834807+0000","last_change":"2024-05-27T00:39:20.576289+0000","last_active":"2024-05-27T00:39:28.834807+0000","last_peered":"2024-05-27T00:39:28.834807+0000","last_clean":"2024-05-27T00:39:28.834807+0000","last_became_active":"2024-05-27T00:39:20.576008+0000","last_became_peered":"2024-05-27T00:39:20.576008+0000","last_unstale":"2024-05-27T00:39:28.834807+0000","last_undegraded":"2024-05-27T00:39:28.834807+0000","last_fullsized":"2024-05-27T00:39:28.834807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610207+0000","last_change":"2024-05-27T00:39:25.964215+0000","last_active":"2024-05-27T00:39:27.610207+0000","last_peered":"2024-05-27T00:39:27.610207+0000","last_clean":"2024-05-27T00:39:27.610207+0000","last_became_active":"2024-05-27T00:39:25.963695+0000","last_became_peered":"2024-05-27T00:39:25.963695+0000","last_unstale":"2024-05-27T00:39:27.610207+0000","last_undegraded":"2024-05-27T00:39:27.610207+0000","last_fullsized":"2024-05-27T00:39:27.610207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:00:27.182513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833721+0000","last_change":"2024-05-27T00:39:21.599920+0000","last_active":"2024-05-27T00:39:28.833721+0000","last_peered":"2024-05-27T00:39:28.833721+0000","last_clean":"2024-05-27T00:39:28.833721+0000","last_became_active":"2024-05-27T00:39:21.597737+0000","last_became_peered":"2024-05-27T00:39:21.597737+0000","last_unstale":"2024-05-27T00:39:28.833721+0000","last_undegraded":"2024-05-27T00:39:28.833721+0000","last_fullsized":"2024-05-27T00:39:28.833721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:38.014686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610101+0000","last_change":"2024-05-27T00:39:20.571178+0000","last_active":"2024-05-27T00:39:27.610101+0000","last_peered":"2024-05-27T00:39:27.610101+0000","last_clean":"2024-05-27T00:39:27.610101+0000","last_became_active":"2024-05-27T00:39:20.570884+0000","last_became_peered":"2024-05-27T00:39:20.570884+0000","last_unstale":"2024-05-27T00:39:27.610101+0000","last_undegraded":"2024-05-27T00:39:27.610101+0000","last_fullsized":"2024-05-27T00:39:27.610101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610204+0000","last_change":"2024-05-27T00:39:25.961014+0000","last_active":"2024-05-27T00:39:27.610204+0000","last_peered":"2024-05-27T00:39:27.610204+0000","last_clean":"2024-05-27T00:39:27.610204+0000","last_became_active":"2024-05-27T00:39:25.960630+0000","last_became_peered":"2024-05-27T00:39:25.960630+0000","last_unstale":"2024-05-27T00:39:27.610204+0000","last_undegraded":"2024-05-27T00:39:27.610204+0000","last_fullsized":"2024-05-27T00:39:27.610204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:53:07.966829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133320+0000","last_change":"2024-05-27T00:39:21.583015+0000","last_active":"2024-05-27T00:39:28.133320+0000","last_peered":"2024-05-27T00:39:28.133320+0000","last_clean":"2024-05-27T00:39:28.133320+0000","last_became_active":"2024-05-27T00:39:21.582722+0000","last_became_peered":"2024-05-27T00:39:21.582722+0000","last_unstale":"2024-05-27T00:39:28.133320+0000","last_undegraded":"2024-05-27T00:39:28.133320+0000","last_fullsized":"2024-05-27T00:39:28.133320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:04.988100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133718+0000","last_change":"2024-05-27T00:39:20.575435+0000","last_active":"2024-05-27T00:39:28.133718+0000","last_peered":"2024-05-27T00:39:28.133718+0000","last_clean":"2024-05-27T00:39:28.133718+0000","last_became_active":"2024-05-27T00:39:20.575144+0000","last_became_peered":"2024-05-27T00:39:20.575144+0000","last_unstale":"2024-05-27T00:39:28.133718+0000","last_undegraded":"2024-05-27T00:39:28.133718+0000","last_fullsized":"2024-05-27T00:39:28.133718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:25:38.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610849+0000","last_change":"2024-05-27T00:39:25.961559+0000","last_active":"2024-05-27T00:39:27.610849+0000","last_peered":"2024-05-27T00:39:27.610849+0000","last_clean":"2024-05-27T00:39:27.610849+0000","last_became_active":"2024-05-27T00:39:25.961155+0000","last_became_peered":"2024-05-27T00:39:25.961155+0000","last_unstale":"2024-05-27T00:39:27.610849+0000","last_undegraded":"2024-05-27T00:39:27.610849+0000","last_fullsized":"2024-05-27T00:39:27.610849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834212+0000","last_change":"2024-05-27T00:39:21.593738+0000","last_active":"2024-05-27T00:39:28.834212+0000","last_peered":"2024-05-27T00:39:28.834212+0000","last_clean":"2024-05-27T00:39:28.834212+0000","last_became_active":"2024-05-27T00:39:21.591298+0000","last_became_peered":"2024-05-27T00:39:21.591298+0000","last_unstale":"2024-05-27T00:39:28.834212+0000","last_undegraded":"2024-05-27T00:39:28.834212+0000","last_fullsized":"2024-05-27T00:39:28.834212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:30.527235+0000","last_change":"2024-05-27T00:39:20.577937+0000","last_active":"2024-05-27T00:39:30.527235+0000","last_peered":"2024-05-27T00:39:30.527235+0000","last_clean":"2024-05-27T00:39:30.527235+0000","last_became_active":"2024-05-27T00:39:20.577630+0000","last_became_peered":"2024-05-27T00:39:20.577630+0000","last_unstale":"2024-05-27T00:39:30.527235+0000","last_undegraded":"2024-05-27T00:39:30.527235+0000","last_fullsized":"2024-05-27T00:39:30.527235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604388+0000","last_change":"2024-05-27T00:39:25.620344+0000","last_active":"2024-05-27T00:39:27.604388+0000","last_peered":"2024-05-27T00:39:27.604388+0000","last_clean":"2024-05-27T00:39:27.604388+0000","last_became_active":"2024-05-27T00:39:25.618090+0000","last_became_peered":"2024-05-27T00:39:25.618090+0000","last_unstale":"2024-05-27T00:39:27.604388+0000","last_undegraded":"2024-05-27T00:39:27.604388+0000","last_fullsized":"2024-05-27T00:39:27.604388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834698+0000","last_change":"2024-05-27T00:39:21.593265+0000","last_active":"2024-05-27T00:39:28.834698+0000","last_peered":"2024-05-27T00:39:28.834698+0000","last_clean":"2024-05-27T00:39:28.834698+0000","last_became_active":"2024-05-27T00:39:21.592984+0000","last_became_peered":"2024-05-27T00:39:21.592984+0000","last_unstale":"2024-05-27T00:39:28.834698+0000","last_undegraded":"2024-05-27T00:39:28.834698+0000","last_fullsized":"2024-05-27T00:39:28.834698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:06.451979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604736+0000","last_change":"2024-05-27T00:39:20.570519+0000","last_active":"2024-05-27T00:39:27.604736+0000","last_peered":"2024-05-27T00:39:27.604736+0000","last_clean":"2024-05-27T00:39:27.604736+0000","last_became_active":"2024-05-27T00:39:20.570230+0000","last_became_peered":"2024-05-27T00:39:20.570230+0000","last_unstale":"2024-05-27T00:39:27.604736+0000","last_undegraded":"2024-05-27T00:39:27.604736+0000","last_fullsized":"2024-05-27T00:39:27.604736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834196+0000","last_change":"2024-05-27T00:39:25.628554+0000","last_active":"2024-05-27T00:39:28.834196+0000","last_peered":"2024-05-27T00:39:28.834196+0000","last_clean":"2024-05-27T00:39:28.834196+0000","last_became_active":"2024-05-27T00:39:25.627971+0000","last_became_peered":"2024-05-27T00:39:25.627971+0000","last_unstale":"2024-05-27T00:39:28.834196+0000","last_undegraded":"2024-05-27T00:39:28.834196+0000","last_fullsized":"2024-05-27T00:39:28.834196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133800+0000","last_change":"2024-05-27T00:39:21.576653+0000","last_active":"2024-05-27T00:39:28.133800+0000","last_peered":"2024-05-27T00:39:28.133800+0000","last_clean":"2024-05-27T00:39:28.133800+0000","last_became_active":"2024-05-27T00:39:21.576383+0000","last_became_peered":"2024-05-27T00:39:21.576383+0000","last_unstale":"2024-05-27T00:39:28.133800+0000","last_undegraded":"2024-05-27T00:39:28.133800+0000","last_fullsized":"2024-05-27T00:39:28.133800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610702+0000","last_change":"2024-05-27T00:39:20.575867+0000","last_active":"2024-05-27T00:39:27.610702+0000","last_peered":"2024-05-27T00:39:27.610702+0000","last_clean":"2024-05-27T00:39:27.610702+0000","last_became_active":"2024-05-27T00:39:20.575557+0000","last_became_peered":"2024-05-27T00:39:20.575557+0000","last_unstale":"2024-05-27T00:39:27.610702+0000","last_undegraded":"2024-05-27T00:39:27.610702+0000","last_fullsized":"2024-05-27T00:39:27.610702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609795+0000","last_change":"2024-05-27T00:39:25.627020+0000","last_active":"2024-05-27T00:39:27.609795+0000","last_peered":"2024-05-27T00:39:27.609795+0000","last_clean":"2024-05-27T00:39:27.609795+0000","last_became_active":"2024-05-27T00:39:25.626629+0000","last_became_peered":"2024-05-27T00:39:25.626629+0000","last_unstale":"2024-05-27T00:39:27.609795+0000","last_undegraded":"2024-05-27T00:39:27.609795+0000","last_fullsized":"2024-05-27T00:39:27.609795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:57.470204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610617+0000","last_change":"2024-05-27T00:39:21.584614+0000","last_active":"2024-05-27T00:39:27.610617+0000","last_peered":"2024-05-27T00:39:27.610617+0000","last_clean":"2024-05-27T00:39:27.610617+0000","last_became_active":"2024-05-27T00:39:21.584349+0000","last_became_peered":"2024-05-27T00:39:21.584349+0000","last_unstale":"2024-05-27T00:39:27.610617+0000","last_undegraded":"2024-05-27T00:39:27.610617+0000","last_fullsized":"2024-05-27T00:39:27.610617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834142+0000","last_change":"2024-05-27T00:39:20.575901+0000","last_active":"2024-05-27T00:39:28.834142+0000","last_peered":"2024-05-27T00:39:28.834142+0000","last_clean":"2024-05-27T00:39:28.834142+0000","last_became_active":"2024-05-27T00:39:20.575521+0000","last_became_peered":"2024-05-27T00:39:20.575521+0000","last_unstale":"2024-05-27T00:39:28.834142+0000","last_undegraded":"2024-05-27T00:39:28.834142+0000","last_fullsized":"2024-05-27T00:39:28.834142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609706+0000","last_change":"2024-05-27T00:39:25.961740+0000","last_active":"2024-05-27T00:39:27.609706+0000","last_peered":"2024-05-27T00:39:27.609706+0000","last_clean":"2024-05-27T00:39:27.609706+0000","last_became_active":"2024-05-27T00:39:25.960524+0000","last_became_peered":"2024-05-27T00:39:25.960524+0000","last_unstale":"2024-05-27T00:39:27.609706+0000","last_undegraded":"2024-05-27T00:39:27.609706+0000","last_fullsized":"2024-05-27T00:39:27.609706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:38:40.065860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133581+0000","last_change":"2024-05-27T00:39:25.961849+0000","last_active":"2024-05-27T00:39:28.133581+0000","last_peered":"2024-05-27T00:39:28.133581+0000","last_clean":"2024-05-27T00:39:28.133581+0000","last_became_active":"2024-05-27T00:39:25.960998+0000","last_became_peered":"2024-05-27T00:39:25.960998+0000","last_unstale":"2024-05-27T00:39:28.133581+0000","last_undegraded":"2024-05-27T00:39:28.133581+0000","last_fullsized":"2024-05-27T00:39:28.133581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604056+0000","last_change":"2024-05-27T00:39:21.590501+0000","last_active":"2024-05-27T00:39:27.604056+0000","last_peered":"2024-05-27T00:39:27.604056+0000","last_clean":"2024-05-27T00:39:27.604056+0000","last_became_active":"2024-05-27T00:39:21.590246+0000","last_became_peered":"2024-05-27T00:39:21.590246+0000","last_unstale":"2024-05-27T00:39:27.604056+0000","last_undegraded":"2024-05-27T00:39:27.604056+0000","last_fullsized":"2024-05-27T00:39:27.604056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609915+0000","last_change":"2024-05-27T00:39:20.574994+0000","last_active":"2024-05-27T00:39:27.609915+0000","last_peered":"2024-05-27T00:39:27.609915+0000","last_clean":"2024-05-27T00:39:27.609915+0000","last_became_active":"2024-05-27T00:39:20.574713+0000","last_became_peered":"2024-05-27T00:39:20.574713+0000","last_unstale":"2024-05-27T00:39:27.609915+0000","last_undegraded":"2024-05-27T00:39:27.609915+0000","last_fullsized":"2024-05-27T00:39:27.609915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835197+0000","last_change":"2024-05-27T00:39:25.629723+0000","last_active":"2024-05-27T00:39:28.835197+0000","last_peered":"2024-05-27T00:39:28.835197+0000","last_clean":"2024-05-27T00:39:28.835197+0000","last_became_active":"2024-05-27T00:39:25.628168+0000","last_became_peered":"2024-05-27T00:39:25.628168+0000","last_unstale":"2024-05-27T00:39:28.835197+0000","last_undegraded":"2024-05-27T00:39:28.835197+0000","last_fullsized":"2024-05-27T00:39:28.835197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609695+0000","last_change":"2024-05-27T00:39:21.591614+0000","last_active":"2024-05-27T00:39:27.609695+0000","last_peered":"2024-05-27T00:39:27.609695+0000","last_clean":"2024-05-27T00:39:27.609695+0000","last_became_active":"2024-05-27T00:39:21.591359+0000","last_became_peered":"2024-05-27T00:39:21.591359+0000","last_unstale":"2024-05-27T00:39:27.609695+0000","last_undegraded":"2024-05-27T00:39:27.609695+0000","last_fullsized":"2024-05-27T00:39:27.609695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610097+0000","last_change":"2024-05-27T00:39:20.575532+0000","last_active":"2024-05-27T00:39:27.610097+0000","last_peered":"2024-05-27T00:39:27.610097+0000","last_clean":"2024-05-27T00:39:27.610097+0000","last_became_active":"2024-05-27T00:39:20.575201+0000","last_became_peered":"2024-05-27T00:39:20.575201+0000","last_unstale":"2024-05-27T00:39:27.610097+0000","last_undegraded":"2024-05-27T00:39:27.610097+0000","last_fullsized":"2024-05-27T00:39:27.610097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834640+0000","last_change":"2024-05-27T00:39:20.577031+0000","last_active":"2024-05-27T00:39:28.834640+0000","last_peered":"2024-05-27T00:39:28.834640+0000","last_clean":"2024-05-27T00:39:28.834640+0000","last_became_active":"2024-05-27T00:39:20.576619+0000","last_became_peered":"2024-05-27T00:39:20.576619+0000","last_unstale":"2024-05-27T00:39:28.834640+0000","last_undegraded":"2024-05-27T00:39:28.834640+0000","last_fullsized":"2024-05-27T00:39:28.834640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133684+0000","last_change":"2024-05-27T00:39:21.580671+0000","last_active":"2024-05-27T00:39:28.133684+0000","last_peered":"2024-05-27T00:39:28.133684+0000","last_clean":"2024-05-27T00:39:28.133684+0000","last_became_active":"2024-05-27T00:39:21.580434+0000","last_became_peered":"2024-05-27T00:39:21.580434+0000","last_unstale":"2024-05-27T00:39:28.133684+0000","last_undegraded":"2024-05-27T00:39:28.133684+0000","last_fullsized":"2024-05-27T00:39:28.133684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832153+0000","last_change":"2024-05-27T00:39:25.624658+0000","last_active":"2024-05-27T00:39:28.832153+0000","last_peered":"2024-05-27T00:39:28.832153+0000","last_clean":"2024-05-27T00:39:28.832153+0000","last_became_active":"2024-05-27T00:39:25.624289+0000","last_became_peered":"2024-05-27T00:39:25.624289+0000","last_unstale":"2024-05-27T00:39:28.832153+0000","last_undegraded":"2024-05-27T00:39:28.832153+0000","last_fullsized":"2024-05-27T00:39:28.832153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:22.409161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609979+0000","last_change":"2024-05-27T00:39:20.572852+0000","last_active":"2024-05-27T00:39:27.609979+0000","last_peered":"2024-05-27T00:39:27.609979+0000","last_clean":"2024-05-27T00:39:27.609979+0000","last_became_active":"2024-05-27T00:39:20.572562+0000","last_became_peered":"2024-05-27T00:39:20.572562+0000","last_unstale":"2024-05-27T00:39:27.609979+0000","last_undegraded":"2024-05-27T00:39:27.609979+0000","last_fullsized":"2024-05-27T00:39:27.609979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833202+0000","last_change":"2024-05-27T00:39:21.587846+0000","last_active":"2024-05-27T00:39:28.833202+0000","last_peered":"2024-05-27T00:39:28.833202+0000","last_clean":"2024-05-27T00:39:28.833202+0000","last_became_active":"2024-05-27T00:39:21.587579+0000","last_became_peered":"2024-05-27T00:39:21.587579+0000","last_unstale":"2024-05-27T00:39:28.833202+0000","last_undegraded":"2024-05-27T00:39:28.833202+0000","last_fullsized":"2024-05-27T00:39:28.833202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:47.382970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610160+0000","last_change":"2024-05-27T00:39:25.628905+0000","last_active":"2024-05-27T00:39:27.610160+0000","last_peered":"2024-05-27T00:39:27.610160+0000","last_clean":"2024-05-27T00:39:27.610160+0000","last_became_active":"2024-05-27T00:39:25.628229+0000","last_became_peered":"2024-05-27T00:39:25.628229+0000","last_unstale":"2024-05-27T00:39:27.610160+0000","last_undegraded":"2024-05-27T00:39:27.610160+0000","last_fullsized":"2024-05-27T00:39:27.610160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:40:24.502057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834905+0000","last_change":"2024-05-27T00:39:21.593542+0000","last_active":"2024-05-27T00:39:28.834905+0000","last_peered":"2024-05-27T00:39:28.834905+0000","last_clean":"2024-05-27T00:39:28.834905+0000","last_became_active":"2024-05-27T00:39:21.592466+0000","last_became_peered":"2024-05-27T00:39:21.592466+0000","last_unstale":"2024-05-27T00:39:28.834905+0000","last_undegraded":"2024-05-27T00:39:28.834905+0000","last_fullsized":"2024-05-27T00:39:28.834905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831986+0000","last_change":"2024-05-27T00:39:21.586207+0000","last_active":"2024-05-27T00:39:28.831986+0000","last_peered":"2024-05-27T00:39:28.831986+0000","last_clean":"2024-05-27T00:39:28.831986+0000","last_became_active":"2024-05-27T00:39:21.585899+0000","last_became_peered":"2024-05-27T00:39:21.585899+0000","last_unstale":"2024-05-27T00:39:28.831986+0000","last_undegraded":"2024-05-27T00:39:28.831986+0000","last_fullsized":"2024-05-27T00:39:28.831986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610077+0000","last_change":"2024-05-27T00:39:21.589304+0000","last_active":"2024-05-27T00:39:27.610077+0000","last_peered":"2024-05-27T00:39:27.610077+0000","last_clean":"2024-05-27T00:39:27.610077+0000","last_became_active":"2024-05-27T00:39:21.588424+0000","last_became_peered":"2024-05-27T00:39:21.588424+0000","last_unstale":"2024-05-27T00:39:27.610077+0000","last_undegraded":"2024-05-27T00:39:27.610077+0000","last_fullsized":"2024-05-27T00:39:27.610077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133594+0000","last_change":"2024-05-27T00:39:21.587527+0000","last_active":"2024-05-27T00:39:28.133594+0000","last_peered":"2024-05-27T00:39:28.133594+0000","last_clean":"2024-05-27T00:39:28.133594+0000","last_became_active":"2024-05-27T00:39:21.587271+0000","last_became_peered":"2024-05-27T00:39:21.587271+0000","last_unstale":"2024-05-27T00:39:28.133594+0000","last_undegraded":"2024-05-27T00:39:28.133594+0000","last_fullsized":"2024-05-27T00:39:28.133594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610312+0000","last_change":"2024-05-27T00:39:21.600567+0000","last_active":"2024-05-27T00:39:27.610312+0000","last_peered":"2024-05-27T00:39:27.610312+0000","last_clean":"2024-05-27T00:39:27.610312+0000","last_became_active":"2024-05-27T00:39:21.600281+0000","last_became_peered":"2024-05-27T00:39:21.600281+0000","last_unstale":"2024-05-27T00:39:27.610312+0000","last_undegraded":"2024-05-27T00:39:27.610312+0000","last_fullsized":"2024-05-27T00:39:27.610312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604571+0000","last_change":"2024-05-27T00:39:21.591200+0000","last_active":"2024-05-27T00:39:27.604571+0000","last_peered":"2024-05-27T00:39:27.604571+0000","last_clean":"2024-05-27T00:39:27.604571+0000","last_became_active":"2024-05-27T00:39:21.590918+0000","last_became_peered":"2024-05-27T00:39:21.590918+0000","last_unstale":"2024-05-27T00:39:27.604571+0000","last_undegraded":"2024-05-27T00:39:27.604571+0000","last_fullsized":"2024-05-27T00:39:27.604571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835139+0000","last_change":"2024-05-27T00:39:21.593373+0000","last_active":"2024-05-27T00:39:28.835139+0000","last_peered":"2024-05-27T00:39:28.835139+0000","last_clean":"2024-05-27T00:39:28.835139+0000","last_became_active":"2024-05-27T00:39:21.590646+0000","last_became_peered":"2024-05-27T00:39:21.590646+0000","last_unstale":"2024-05-27T00:39:28.835139+0000","last_undegraded":"2024-05-27T00:39:28.835139+0000","last_fullsized":"2024-05-27T00:39:28.835139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610260+0000","last_change":"2024-05-27T00:39:21.600531+0000","last_active":"2024-05-27T00:39:27.610260+0000","last_peered":"2024-05-27T00:39:27.610260+0000","last_clean":"2024-05-27T00:39:27.610260+0000","last_became_active":"2024-05-27T00:39:21.600236+0000","last_became_peered":"2024-05-27T00:39:21.600236+0000","last_unstale":"2024-05-27T00:39:27.610260+0000","last_undegraded":"2024-05-27T00:39:27.610260+0000","last_fullsized":"2024-05-27T00:39:27.610260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609848+0000","last_change":"2024-05-27T00:39:21.588532+0000","last_active":"2024-05-27T00:39:27.609848+0000","last_peered":"2024-05-27T00:39:27.609848+0000","last_clean":"2024-05-27T00:39:27.609848+0000","last_became_active":"2024-05-27T00:39:21.587475+0000","last_became_peered":"2024-05-27T00:39:21.587475+0000","last_unstale":"2024-05-27T00:39:27.609848+0000","last_undegraded":"2024-05-27T00:39:27.609848+0000","last_fullsized":"2024-05-27T00:39:27.609848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609903+0000","last_change":"2024-05-27T00:39:21.601094+0000","last_active":"2024-05-27T00:39:27.609903+0000","last_peered":"2024-05-27T00:39:27.609903+0000","last_clean":"2024-05-27T00:39:27.609903+0000","last_became_active":"2024-05-27T00:39:21.600345+0000","last_became_peered":"2024-05-27T00:39:21.600345+0000","last_unstale":"2024-05-27T00:39:27.609903+0000","last_undegraded":"2024-05-27T00:39:27.609903+0000","last_fullsized":"2024-05-27T00:39:27.609903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609931+0000","last_change":"2024-05-27T00:39:21.583984+0000","last_active":"2024-05-27T00:39:27.609931+0000","last_peered":"2024-05-27T00:39:27.609931+0000","last_clean":"2024-05-27T00:39:27.609931+0000","last_became_active":"2024-05-27T00:39:21.583601+0000","last_became_peered":"2024-05-27T00:39:21.583601+0000","last_unstale":"2024-05-27T00:39:27.609931+0000","last_undegraded":"2024-05-27T00:39:27.609931+0000","last_fullsized":"2024-05-27T00:39:27.609931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133437+0000","last_change":"2024-05-27T00:39:21.583713+0000","last_active":"2024-05-27T00:39:28.133437+0000","last_peered":"2024-05-27T00:39:28.133437+0000","last_clean":"2024-05-27T00:39:28.133437+0000","last_became_active":"2024-05-27T00:39:21.583367+0000","last_became_peered":"2024-05-27T00:39:21.583367+0000","last_unstale":"2024-05-27T00:39:28.133437+0000","last_undegraded":"2024-05-27T00:39:28.133437+0000","last_fullsized":"2024-05-27T00:39:28.133437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834031+0000","last_change":"2024-05-27T00:39:21.597145+0000","last_active":"2024-05-27T00:39:28.834031+0000","last_peered":"2024-05-27T00:39:28.834031+0000","last_clean":"2024-05-27T00:39:28.834031+0000","last_became_active":"2024-05-27T00:39:21.596737+0000","last_became_peered":"2024-05-27T00:39:21.596737+0000","last_unstale":"2024-05-27T00:39:28.834031+0000","last_undegraded":"2024-05-27T00:39:28.834031+0000","last_fullsized":"2024-05-27T00:39:28.834031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834671+0000","last_change":"2024-05-27T00:39:21.586729+0000","last_active":"2024-05-27T00:39:28.834671+0000","last_peered":"2024-05-27T00:39:28.834671+0000","last_clean":"2024-05-27T00:39:28.834671+0000","last_became_active":"2024-05-27T00:39:21.586428+0000","last_became_peered":"2024-05-27T00:39:21.586428+0000","last_unstale":"2024-05-27T00:39:28.834671+0000","last_undegraded":"2024-05-27T00:39:28.834671+0000","last_fullsized":"2024-05-27T00:39:28.834671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832018+0000","last_change":"2024-05-27T00:39:21.591955+0000","last_active":"2024-05-27T00:39:28.832018+0000","last_peered":"2024-05-27T00:39:28.832018+0000","last_clean":"2024-05-27T00:39:28.832018+0000","last_became_active":"2024-05-27T00:39:21.591645+0000","last_became_peered":"2024-05-27T00:39:21.591645+0000","last_unstale":"2024-05-27T00:39:28.832018+0000","last_undegraded":"2024-05-27T00:39:28.832018+0000","last_fullsized":"2024-05-27T00:39:28.832018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610938+0000","last_change":"2024-05-27T00:39:21.582309+0000","last_active":"2024-05-27T00:39:27.610938+0000","last_peered":"2024-05-27T00:39:27.610938+0000","last_clean":"2024-05-27T00:39:27.610938+0000","last_became_active":"2024-05-27T00:39:21.582029+0000","last_became_peered":"2024-05-27T00:39:21.582029+0000","last_unstale":"2024-05-27T00:39:27.610938+0000","last_undegraded":"2024-05-27T00:39:27.610938+0000","last_fullsized":"2024-05-27T00:39:27.610938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609792+0000","last_change":"2024-05-27T00:39:21.583269+0000","last_active":"2024-05-27T00:39:27.609792+0000","last_peered":"2024-05-27T00:39:27.609792+0000","last_clean":"2024-05-27T00:39:27.609792+0000","last_became_active":"2024-05-27T00:39:21.582827+0000","last_became_peered":"2024-05-27T00:39:21.582827+0000","last_unstale":"2024-05-27T00:39:27.609792+0000","last_undegraded":"2024-05-27T00:39:27.609792+0000","last_fullsized":"2024-05-27T00:39:27.609792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604091+0000","last_change":"2024-05-27T00:39:21.600540+0000","last_active":"2024-05-27T00:39:27.604091+0000","last_peered":"2024-05-27T00:39:27.604091+0000","last_clean":"2024-05-27T00:39:27.604091+0000","last_became_active":"2024-05-27T00:39:21.600270+0000","last_became_peered":"2024-05-27T00:39:21.600270+0000","last_unstale":"2024-05-27T00:39:27.604091+0000","last_undegraded":"2024-05-27T00:39:27.604091+0000","last_fullsized":"2024-05-27T00:39:27.604091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133315+0000","last_change":"2024-05-27T00:39:21.587064+0000","last_active":"2024-05-27T00:39:28.133315+0000","last_peered":"2024-05-27T00:39:28.133315+0000","last_clean":"2024-05-27T00:39:28.133315+0000","last_became_active":"2024-05-27T00:39:21.586759+0000","last_became_peered":"2024-05-27T00:39:21.586759+0000","last_unstale":"2024-05-27T00:39:28.133315+0000","last_undegraded":"2024-05-27T00:39:28.133315+0000","last_fullsized":"2024-05-27T00:39:28.133315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:18.374156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609806+0000","last_change":"2024-05-27T00:39:21.594918+0000","last_active":"2024-05-27T00:39:27.609806+0000","last_peered":"2024-05-27T00:39:27.609806+0000","last_clean":"2024-05-27T00:39:27.609806+0000","last_became_active":"2024-05-27T00:39:21.594670+0000","last_became_peered":"2024-05-27T00:39:21.594670+0000","last_unstale":"2024-05-27T00:39:27.609806+0000","last_undegraded":"2024-05-27T00:39:27.609806+0000","last_fullsized":"2024-05-27T00:39:27.609806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834590+0000","last_change":"2024-05-27T00:39:21.600525+0000","last_active":"2024-05-27T00:39:28.834590+0000","last_peered":"2024-05-27T00:39:28.834590+0000","last_clean":"2024-05-27T00:39:28.834590+0000","last_became_active":"2024-05-27T00:39:21.600012+0000","last_became_peered":"2024-05-27T00:39:21.600012+0000","last_unstale":"2024-05-27T00:39:28.834590+0000","last_undegraded":"2024-05-27T00:39:28.834590+0000","last_fullsized":"2024-05-27T00:39:28.834590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834173+0000","last_change":"2024-05-27T00:39:21.596158+0000","last_active":"2024-05-27T00:39:28.834173+0000","last_peered":"2024-05-27T00:39:28.834173+0000","last_clean":"2024-05-27T00:39:28.834173+0000","last_became_active":"2024-05-27T00:39:21.595655+0000","last_became_peered":"2024-05-27T00:39:21.595655+0000","last_unstale":"2024-05-27T00:39:28.834173+0000","last_undegraded":"2024-05-27T00:39:28.834173+0000","last_fullsized":"2024-05-27T00:39:28.834173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835071+0000","last_change":"2024-05-27T00:39:21.594957+0000","last_active":"2024-05-27T00:39:28.835071+0000","last_peered":"2024-05-27T00:39:28.835071+0000","last_clean":"2024-05-27T00:39:28.835071+0000","last_became_active":"2024-05-27T00:39:21.594682+0000","last_became_peered":"2024-05-27T00:39:21.594682+0000","last_unstale":"2024-05-27T00:39:28.835071+0000","last_undegraded":"2024-05-27T00:39:28.835071+0000","last_fullsized":"2024-05-27T00:39:28.835071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833653+0000","last_change":"2024-05-27T00:39:21.589852+0000","last_active":"2024-05-27T00:39:28.833653+0000","last_peered":"2024-05-27T00:39:28.833653+0000","last_clean":"2024-05-27T00:39:28.833653+0000","last_became_active":"2024-05-27T00:39:21.589514+0000","last_became_peered":"2024-05-27T00:39:21.589514+0000","last_unstale":"2024-05-27T00:39:28.833653+0000","last_undegraded":"2024-05-27T00:39:28.833653+0000","last_fullsized":"2024-05-27T00:39:28.833653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:08.796084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609583+0000","last_change":"2024-05-27T00:39:21.586007+0000","last_active":"2024-05-27T00:39:27.609583+0000","last_peered":"2024-05-27T00:39:27.609583+0000","last_clean":"2024-05-27T00:39:27.609583+0000","last_became_active":"2024-05-27T00:39:21.585708+0000","last_became_peered":"2024-05-27T00:39:21.585708+0000","last_unstale":"2024-05-27T00:39:27.609583+0000","last_undegraded":"2024-05-27T00:39:27.609583+0000","last_fullsized":"2024-05-27T00:39:27.609583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832658+0000","last_change":"2024-05-27T00:39:21.596377+0000","last_active":"2024-05-27T00:39:28.832658+0000","last_peered":"2024-05-27T00:39:28.832658+0000","last_clean":"2024-05-27T00:39:28.832658+0000","last_became_active":"2024-05-27T00:39:21.596055+0000","last_became_peered":"2024-05-27T00:39:21.596055+0000","last_unstale":"2024-05-27T00:39:28.832658+0000","last_undegraded":"2024-05-27T00:39:28.832658+0000","last_fullsized":"2024-05-27T00:39:28.832658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609929+0000","last_change":"2024-05-27T00:39:21.583846+0000","last_active":"2024-05-27T00:39:27.609929+0000","last_peered":"2024-05-27T00:39:27.609929+0000","last_clean":"2024-05-27T00:39:27.609929+0000","last_became_active":"2024-05-27T00:39:21.583284+0000","last_became_peered":"2024-05-27T00:39:21.583284+0000","last_unstale":"2024-05-27T00:39:27.609929+0000","last_undegraded":"2024-05-27T00:39:27.609929+0000","last_fullsized":"2024-05-27T00:39:27.609929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833536+0000","last_change":"2024-05-27T00:39:21.586873+0000","last_active":"2024-05-27T00:39:28.833536+0000","last_peered":"2024-05-27T00:39:28.833536+0000","last_clean":"2024-05-27T00:39:28.833536+0000","last_became_active":"2024-05-27T00:39:21.586593+0000","last_became_peered":"2024-05-27T00:39:21.586593+0000","last_unstale":"2024-05-27T00:39:28.833536+0000","last_undegraded":"2024-05-27T00:39:28.833536+0000","last_fullsized":"2024-05-27T00:39:28.833536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610399+0000","last_change":"2024-05-27T00:39:21.592963+0000","last_active":"2024-05-27T00:39:27.610399+0000","last_peered":"2024-05-27T00:39:27.610399+0000","last_clean":"2024-05-27T00:39:27.610399+0000","last_became_active":"2024-05-27T00:39:21.588049+0000","last_became_peered":"2024-05-27T00:39:21.588049+0000","last_unstale":"2024-05-27T00:39:27.610399+0000","last_undegraded":"2024-05-27T00:39:27.610399+0000","last_fullsized":"2024-05-27T00:39:27.610399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133490+0000","last_change":"2024-05-27T00:39:21.588239+0000","last_active":"2024-05-27T00:39:28.133490+0000","last_peered":"2024-05-27T00:39:28.133490+0000","last_clean":"2024-05-27T00:39:28.133490+0000","last_became_active":"2024-05-27T00:39:21.587936+0000","last_became_peered":"2024-05-27T00:39:21.587936+0000","last_unstale":"2024-05-27T00:39:28.133490+0000","last_undegraded":"2024-05-27T00:39:28.133490+0000","last_fullsized":"2024-05-27T00:39:28.133490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604778+0000","last_change":"2024-05-27T00:39:21.591553+0000","last_active":"2024-05-27T00:39:27.604778+0000","last_peered":"2024-05-27T00:39:27.604778+0000","last_clean":"2024-05-27T00:39:27.604778+0000","last_became_active":"2024-05-27T00:39:21.591286+0000","last_became_peered":"2024-05-27T00:39:21.591286+0000","last_unstale":"2024-05-27T00:39:27.604778+0000","last_undegraded":"2024-05-27T00:39:27.604778+0000","last_fullsized":"2024-05-27T00:39:27.604778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833459+0000","last_change":"2024-05-27T00:39:21.591695+0000","last_active":"2024-05-27T00:39:28.833459+0000","last_peered":"2024-05-27T00:39:28.833459+0000","last_clean":"2024-05-27T00:39:28.833459+0000","last_became_active":"2024-05-27T00:39:21.591384+0000","last_became_peered":"2024-05-27T00:39:21.591384+0000","last_unstale":"2024-05-27T00:39:28.833459+0000","last_undegraded":"2024-05-27T00:39:28.833459+0000","last_fullsized":"2024-05-27T00:39:28.833459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:39:39.600 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:39:39.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 Processor -- start 2024-05-27T00:39:39.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 -- start start 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fed70161e50 0x7fed701622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed700d2c10 0x7fed700d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed7006cc70 con 0x7fed700b3630 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fed700d35c0 con 0x7fed700d2c10 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.734+0000 7fed76790640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fed700d1e50 con 0x7fed70161e50 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed74d2d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed700d2c10 0x7fed700d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed6ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fed700b3630 0x7fed700b72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35452/0 (socket says 172.21.15.111:35452) 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed6ffff640 1 -- 172.21.15.111:0/1815816253 learned_addr learned my addr 172.21.15.111:0/1815816253 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed74d2d640 1 -- 172.21.15.111:0/1815816253 >> v2:172.21.15.153:3300/0 conn(0x7fed70161e50 msgr2=0x7fed701622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:39.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed6f7fe640 1 --2- 172.21.15.111:0/1815816253 >> v2:172.21.15.153:3300/0 conn(0x7fed70161e50 0x7fed701622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed6effd640 1 -- 172.21.15.111:0/1815816253 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323350613 0 0) 0x7fed7006cc70 con 0x7fed700b3630 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed74d2d640 1 --2- 172.21.15.111:0/1815816253 >> v2:172.21.15.153:3300/0 conn(0x7fed70161e50 0x7fed701622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed74d2d640 1 -- 172.21.15.111:0/1815816253 >> v1:172.21.15.111:6789/0 conn(0x7fed700b3630 legacy=0x7fed700b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.735+0000 7fed74d2d640 1 -- 172.21.15.111:0/1815816253 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed700d2130 con 0x7fed700d2c10 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed74d2d640 1 --2- 172.21.15.111:0/1815816253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700d2c10 0x7fed700d3080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fed7006d5d0 tx=0x7fed6802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e23d4816c4493c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed6effd640 1 -- 172.21.15.111:0/1815816253 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007740 con 0x7fed700d2c10 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed6effd640 1 -- 172.21.15.111:0/1815816253 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed680078e0 con 0x7fed700d2c10 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed6effd640 1 -- 172.21.15.111:0/1815816253 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007cb0 con 0x7fed700d2c10 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 -- 172.21.15.111:0/1815816253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700d2c10 msgr2=0x7fed700d3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 --2- 172.21.15.111:0/1815816253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700d2c10 0x7fed700d3080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fed7006d5d0 tx=0x7fed6802a680 comp rx=0 tx=0).stop 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 -- 172.21.15.111:0/1815816253 shutdown_connections 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 --2- 172.21.15.111:0/1815816253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700d2c10 0x7fed700d3080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 --2- 172.21.15.111:0/1815816253 >> v2:172.21.15.153:3300/0 conn(0x7fed70161e50 0x7fed701622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 -- 172.21.15.111:0/1815816253 >> 172.21.15.111:0/1815816253 conn(0x7fed700d42c0 msgr2=0x7fed700d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:39.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 -- 172.21.15.111:0/1815816253 shutdown_connections 2024-05-27T00:39:39.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.736+0000 7fed76790640 1 -- 172.21.15.111:0/1815816253 wait complete. 2024-05-27T00:39:39.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.737+0000 7fed76790640 1 Processor -- start 2024-05-27T00:39:39.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 -- start start 2024-05-27T00:39:39.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 0x7fed701f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed700d2c10 0x7fed70152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed70161e50 0x7fed700cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed700b79b0 con 0x7fed700b3630 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fed700d26f0 con 0x7fed70161e50 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.738+0000 7fed76790640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fed7006bfb0 con 0x7fed700d2c10 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 0x7fed701f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 0x7fed701f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56870/0 (socket says 172.21.15.111:56870) 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 -- 172.21.15.111:0/2827309240 learned_addr learned my addr 172.21.15.111:0/2827309240 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed74d2d640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed70161e50 0x7fed700cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6f7fe640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed700d2c10 0x7fed70152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 -- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed700d2c10 msgr2=0x7fed70152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed700d2c10 0x7fed70152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 -- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed70161e50 msgr2=0x7fed700cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed70161e50 0x7fed700cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 -- 172.21.15.111:0/2827309240 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed701f42a0 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed74d2d640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed70161e50 0x7fed700cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6f7fe640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed700d2c10 0x7fed70152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6ffff640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 0x7fed701f3d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c4d0 tx=0x7fed6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60018070 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed60007500 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60004a20 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed700ccb00 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.739+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed700cd0c0 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.740+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed70068680 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.740+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed6001b070 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.741+0000 7fed6cff9640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fed5c06be60 0x7fed5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.741+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fed6008a4e0 con 0x7fed700b3630 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.741+0000 7fed6f7fe640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fed5c06be60 0x7fed5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:39.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.742+0000 7fed6f7fe640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fed5c06be60 0x7fed5c06e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed700ac120 tx=0x7fed64008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:39.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.745+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed60058b30 con 0x7fed700b3630 2024-05-27T00:39:39.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.913+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed700cef10 con 0x7fed5c06be60 2024-05-27T00:39:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.922+0000 7fed6cff9640 1 -- 172.21.15.111:0/2827309240 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fed700cef10 con 0x7fed5c06be60 2024-05-27T00:39:39.926 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:39.928 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fed5c06be60 msgr2=0x7fed5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fed5c06be60 0x7fed5c06e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed700ac120 tx=0x7fed64008040 comp rx=0 tx=0).stop 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 msgr2=0x7fed701f3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 0x7fed701f3d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed6000c4d0 tx=0x7fed6000c990 comp rx=0 tx=0).stop 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 shutdown_connections 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fed5c06be60 0x7fed5c06e320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed70161e50 0x7fed700cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fed700d2c10 0x7fed70152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 --2- 172.21.15.111:0/2827309240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed700b3630 0x7fed701f3d60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 >> 172.21.15.111:0/2827309240 conn(0x7fed700d42c0 msgr2=0x7fed700d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 shutdown_connections 2024-05-27T00:39:39.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:39.925+0000 7fed76790640 1 -- 172.21.15.111:0/2827309240 wait complete. 2024-05-27T00:39:39.937 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-27T00:39:38.928518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2048876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.320081"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133844+0000","last_change":"2024-05-27T00:39:20.574932+0000","last_active":"2024-05-27T00:39:28.133844+0000","last_peered":"2024-05-27T00:39:28.133844+0000","last_clean":"2024-05-27T00:39:28.133844+0000","last_became_active":"2024-05-27T00:39:20.574617+0000","last_became_peered":"2024-05-27T00:39:20.574617+0000","last_unstale":"2024-05-27T00:39:28.133844+0000","last_undegraded":"2024-05-27T00:39:28.133844+0000","last_fullsized":"2024-05-27T00:39:28.133844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610219+0000","last_change":"2024-05-27T00:39:21.594550+0000","last_active":"2024-05-27T00:39:27.610219+0000","last_peered":"2024-05-27T00:39:27.610219+0000","last_clean":"2024-05-27T00:39:27.610219+0000","last_became_active":"2024-05-27T00:39:21.594296+0000","last_became_peered":"2024-05-27T00:39:21.594296+0000","last_unstale":"2024-05-27T00:39:27.610219+0000","last_undegraded":"2024-05-27T00:39:27.610219+0000","last_fullsized":"2024-05-27T00:39:27.610219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609737+0000","last_change":"2024-05-27T00:39:25.965194+0000","last_active":"2024-05-27T00:39:27.609737+0000","last_peered":"2024-05-27T00:39:27.609737+0000","last_clean":"2024-05-27T00:39:27.609737+0000","last_became_active":"2024-05-27T00:39:25.964829+0000","last_became_peered":"2024-05-27T00:39:25.964829+0000","last_unstale":"2024-05-27T00:39:27.609737+0000","last_undegraded":"2024-05-27T00:39:27.609737+0000","last_fullsized":"2024-05-27T00:39:27.609737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:38.727603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610308+0000","last_change":"2024-05-27T00:39:20.569739+0000","last_active":"2024-05-27T00:39:27.610308+0000","last_peered":"2024-05-27T00:39:27.610308+0000","last_clean":"2024-05-27T00:39:27.610308+0000","last_became_active":"2024-05-27T00:39:20.569481+0000","last_became_peered":"2024-05-27T00:39:20.569481+0000","last_unstale":"2024-05-27T00:39:27.610308+0000","last_undegraded":"2024-05-27T00:39:27.610308+0000","last_fullsized":"2024-05-27T00:39:27.610308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133753+0000","last_change":"2024-05-27T00:39:21.576759+0000","last_active":"2024-05-27T00:39:28.133753+0000","last_peered":"2024-05-27T00:39:28.133753+0000","last_clean":"2024-05-27T00:39:28.133753+0000","last_became_active":"2024-05-27T00:39:21.576492+0000","last_became_peered":"2024-05-27T00:39:21.576492+0000","last_unstale":"2024-05-27T00:39:28.133753+0000","last_undegraded":"2024-05-27T00:39:28.133753+0000","last_fullsized":"2024-05-27T00:39:28.133753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833532+0000","last_change":"2024-05-27T00:39:25.626523+0000","last_active":"2024-05-27T00:39:28.833532+0000","last_peered":"2024-05-27T00:39:28.833532+0000","last_clean":"2024-05-27T00:39:28.833532+0000","last_became_active":"2024-05-27T00:39:25.626132+0000","last_became_peered":"2024-05-27T00:39:25.626132+0000","last_unstale":"2024-05-27T00:39:28.833532+0000","last_undegraded":"2024-05-27T00:39:28.833532+0000","last_fullsized":"2024-05-27T00:39:28.833532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834538+0000","last_change":"2024-05-27T00:39:20.577889+0000","last_active":"2024-05-27T00:39:28.834538+0000","last_peered":"2024-05-27T00:39:28.834538+0000","last_clean":"2024-05-27T00:39:28.834538+0000","last_became_active":"2024-05-27T00:39:20.577565+0000","last_became_peered":"2024-05-27T00:39:20.577565+0000","last_unstale":"2024-05-27T00:39:28.834538+0000","last_undegraded":"2024-05-27T00:39:28.834538+0000","last_fullsized":"2024-05-27T00:39:28.834538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:12:22.836110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610522+0000","last_change":"2024-05-27T00:39:21.580044+0000","last_active":"2024-05-27T00:39:27.610522+0000","last_peered":"2024-05-27T00:39:27.610522+0000","last_clean":"2024-05-27T00:39:27.610522+0000","last_became_active":"2024-05-27T00:39:21.579770+0000","last_became_peered":"2024-05-27T00:39:21.579770+0000","last_unstale":"2024-05-27T00:39:27.610522+0000","last_undegraded":"2024-05-27T00:39:27.610522+0000","last_fullsized":"2024-05-27T00:39:27.610522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833971+0000","last_change":"2024-05-27T00:39:25.627365+0000","last_active":"2024-05-27T00:39:28.833971+0000","last_peered":"2024-05-27T00:39:28.833971+0000","last_clean":"2024-05-27T00:39:28.833971+0000","last_became_active":"2024-05-27T00:39:25.626973+0000","last_became_peered":"2024-05-27T00:39:25.626973+0000","last_unstale":"2024-05-27T00:39:28.833971+0000","last_undegraded":"2024-05-27T00:39:28.833971+0000","last_fullsized":"2024-05-27T00:39:28.833971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833143+0000","last_change":"2024-05-27T00:39:20.577815+0000","last_active":"2024-05-27T00:39:28.833143+0000","last_peered":"2024-05-27T00:39:28.833143+0000","last_clean":"2024-05-27T00:39:28.833143+0000","last_became_active":"2024-05-27T00:39:20.577495+0000","last_became_peered":"2024-05-27T00:39:20.577495+0000","last_unstale":"2024-05-27T00:39:28.833143+0000","last_undegraded":"2024-05-27T00:39:28.833143+0000","last_fullsized":"2024-05-27T00:39:28.833143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:05.446553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610465+0000","last_change":"2024-05-27T00:39:21.576226+0000","last_active":"2024-05-27T00:39:27.610465+0000","last_peered":"2024-05-27T00:39:27.610465+0000","last_clean":"2024-05-27T00:39:27.610465+0000","last_became_active":"2024-05-27T00:39:21.575940+0000","last_became_peered":"2024-05-27T00:39:21.575940+0000","last_unstale":"2024-05-27T00:39:27.610465+0000","last_undegraded":"2024-05-27T00:39:27.610465+0000","last_fullsized":"2024-05-27T00:39:27.610465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833014+0000","last_change":"2024-05-27T00:39:25.623493+0000","last_active":"2024-05-27T00:39:28.833014+0000","last_peered":"2024-05-27T00:39:28.833014+0000","last_clean":"2024-05-27T00:39:28.833014+0000","last_became_active":"2024-05-27T00:39:25.623101+0000","last_became_peered":"2024-05-27T00:39:25.623101+0000","last_unstale":"2024-05-27T00:39:28.833014+0000","last_undegraded":"2024-05-27T00:39:28.833014+0000","last_fullsized":"2024-05-27T00:39:28.833014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:12.831089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834471+0000","last_change":"2024-05-27T00:39:21.594839+0000","last_active":"2024-05-27T00:39:28.834471+0000","last_peered":"2024-05-27T00:39:28.834471+0000","last_clean":"2024-05-27T00:39:28.834471+0000","last_became_active":"2024-05-27T00:39:21.594428+0000","last_became_peered":"2024-05-27T00:39:21.594428+0000","last_unstale":"2024-05-27T00:39:28.834471+0000","last_undegraded":"2024-05-27T00:39:28.834471+0000","last_fullsized":"2024-05-27T00:39:28.834471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833832+0000","last_change":"2024-05-27T00:39:20.579815+0000","last_active":"2024-05-27T00:39:28.833832+0000","last_peered":"2024-05-27T00:39:28.833832+0000","last_clean":"2024-05-27T00:39:28.833832+0000","last_became_active":"2024-05-27T00:39:20.579507+0000","last_became_peered":"2024-05-27T00:39:20.579507+0000","last_unstale":"2024-05-27T00:39:28.833832+0000","last_undegraded":"2024-05-27T00:39:28.833832+0000","last_fullsized":"2024-05-27T00:39:28.833832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:17.368730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833761+0000","last_change":"2024-05-27T00:39:25.625823+0000","last_active":"2024-05-27T00:39:28.833761+0000","last_peered":"2024-05-27T00:39:28.833761+0000","last_clean":"2024-05-27T00:39:28.833761+0000","last_became_active":"2024-05-27T00:39:25.625378+0000","last_became_peered":"2024-05-27T00:39:25.625378+0000","last_unstale":"2024-05-27T00:39:28.833761+0000","last_undegraded":"2024-05-27T00:39:28.833761+0000","last_fullsized":"2024-05-27T00:39:28.833761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609891+0000","last_change":"2024-05-27T00:39:21.591774+0000","last_active":"2024-05-27T00:39:27.609891+0000","last_peered":"2024-05-27T00:39:27.609891+0000","last_clean":"2024-05-27T00:39:27.609891+0000","last_became_active":"2024-05-27T00:39:21.591413+0000","last_became_peered":"2024-05-27T00:39:21.591413+0000","last_unstale":"2024-05-27T00:39:27.609891+0000","last_undegraded":"2024-05-27T00:39:27.609891+0000","last_fullsized":"2024-05-27T00:39:27.609891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604467+0000","last_change":"2024-05-27T00:39:20.562227+0000","last_active":"2024-05-27T00:39:27.604467+0000","last_peered":"2024-05-27T00:39:27.604467+0000","last_clean":"2024-05-27T00:39:27.604467+0000","last_became_active":"2024-05-27T00:39:20.561938+0000","last_became_peered":"2024-05-27T00:39:20.561938+0000","last_unstale":"2024-05-27T00:39:27.604467+0000","last_undegraded":"2024-05-27T00:39:27.604467+0000","last_fullsized":"2024-05-27T00:39:27.604467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834385+0000","last_change":"2024-05-27T00:39:25.628730+0000","last_active":"2024-05-27T00:39:28.834385+0000","last_peered":"2024-05-27T00:39:28.834385+0000","last_clean":"2024-05-27T00:39:28.834385+0000","last_became_active":"2024-05-27T00:39:25.628352+0000","last_became_peered":"2024-05-27T00:39:25.628352+0000","last_unstale":"2024-05-27T00:39:28.834385+0000","last_undegraded":"2024-05-27T00:39:28.834385+0000","last_fullsized":"2024-05-27T00:39:28.834385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609789+0000","last_change":"2024-05-27T00:39:21.585438+0000","last_active":"2024-05-27T00:39:27.609789+0000","last_peered":"2024-05-27T00:39:27.609789+0000","last_clean":"2024-05-27T00:39:27.609789+0000","last_became_active":"2024-05-27T00:39:21.585062+0000","last_became_peered":"2024-05-27T00:39:21.585062+0000","last_unstale":"2024-05-27T00:39:27.609789+0000","last_undegraded":"2024-05-27T00:39:27.609789+0000","last_fullsized":"2024-05-27T00:39:27.609789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604165+0000","last_change":"2024-05-27T00:39:20.570857+0000","last_active":"2024-05-27T00:39:27.604165+0000","last_peered":"2024-05-27T00:39:27.604165+0000","last_clean":"2024-05-27T00:39:27.604165+0000","last_became_active":"2024-05-27T00:39:20.570564+0000","last_became_peered":"2024-05-27T00:39:20.570564+0000","last_unstale":"2024-05-27T00:39:27.604165+0000","last_undegraded":"2024-05-27T00:39:27.604165+0000","last_fullsized":"2024-05-27T00:39:27.604165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605057+0000","last_change":"2024-05-27T00:39:25.963988+0000","last_active":"2024-05-27T00:39:27.605057+0000","last_peered":"2024-05-27T00:39:27.605057+0000","last_clean":"2024-05-27T00:39:27.605057+0000","last_became_active":"2024-05-27T00:39:25.963532+0000","last_became_peered":"2024-05-27T00:39:25.963532+0000","last_unstale":"2024-05-27T00:39:27.605057+0000","last_undegraded":"2024-05-27T00:39:27.605057+0000","last_fullsized":"2024-05-27T00:39:27.605057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833654+0000","last_change":"2024-05-27T00:39:21.590869+0000","last_active":"2024-05-27T00:39:28.833654+0000","last_peered":"2024-05-27T00:39:28.833654+0000","last_clean":"2024-05-27T00:39:28.833654+0000","last_became_active":"2024-05-27T00:39:21.584136+0000","last_became_peered":"2024-05-27T00:39:21.584136+0000","last_unstale":"2024-05-27T00:39:28.833654+0000","last_undegraded":"2024-05-27T00:39:28.833654+0000","last_fullsized":"2024-05-27T00:39:28.833654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:48.264046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832474+0000","last_change":"2024-05-27T00:39:20.579097+0000","last_active":"2024-05-27T00:39:28.832474+0000","last_peered":"2024-05-27T00:39:28.832474+0000","last_clean":"2024-05-27T00:39:28.832474+0000","last_became_active":"2024-05-27T00:39:20.578796+0000","last_became_peered":"2024-05-27T00:39:20.578796+0000","last_unstale":"2024-05-27T00:39:28.832474+0000","last_undegraded":"2024-05-27T00:39:28.832474+0000","last_fullsized":"2024-05-27T00:39:28.832474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833500+0000","last_change":"2024-05-27T00:39:25.628322+0000","last_active":"2024-05-27T00:39:28.833500+0000","last_peered":"2024-05-27T00:39:28.833500+0000","last_clean":"2024-05-27T00:39:28.833500+0000","last_became_active":"2024-05-27T00:39:25.627878+0000","last_became_peered":"2024-05-27T00:39:25.627878+0000","last_unstale":"2024-05-27T00:39:28.833500+0000","last_undegraded":"2024-05-27T00:39:28.833500+0000","last_fullsized":"2024-05-27T00:39:28.833500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:42.049691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133677+0000","last_change":"2024-05-27T00:39:21.581752+0000","last_active":"2024-05-27T00:39:28.133677+0000","last_peered":"2024-05-27T00:39:28.133677+0000","last_clean":"2024-05-27T00:39:28.133677+0000","last_became_active":"2024-05-27T00:39:21.581488+0000","last_became_peered":"2024-05-27T00:39:21.581488+0000","last_unstale":"2024-05-27T00:39:28.133677+0000","last_undegraded":"2024-05-27T00:39:28.133677+0000","last_fullsized":"2024-05-27T00:39:28.133677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835187+0000","last_change":"2024-05-27T00:39:20.578119+0000","last_active":"2024-05-27T00:39:28.835187+0000","last_peered":"2024-05-27T00:39:28.835187+0000","last_clean":"2024-05-27T00:39:28.835187+0000","last_became_active":"2024-05-27T00:39:20.577830+0000","last_became_peered":"2024-05-27T00:39:20.577830+0000","last_unstale":"2024-05-27T00:39:28.835187+0000","last_undegraded":"2024-05-27T00:39:28.835187+0000","last_fullsized":"2024-05-27T00:39:28.835187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832957+0000","last_change":"2024-05-27T00:39:25.965993+0000","last_active":"2024-05-27T00:39:28.832957+0000","last_peered":"2024-05-27T00:39:28.832957+0000","last_clean":"2024-05-27T00:39:28.832957+0000","last_became_active":"2024-05-27T00:39:25.965384+0000","last_became_peered":"2024-05-27T00:39:25.965384+0000","last_unstale":"2024-05-27T00:39:28.832957+0000","last_undegraded":"2024-05-27T00:39:28.832957+0000","last_fullsized":"2024-05-27T00:39:28.832957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610354+0000","last_change":"2024-05-27T00:39:21.591396+0000","last_active":"2024-05-27T00:39:27.610354+0000","last_peered":"2024-05-27T00:39:27.610354+0000","last_clean":"2024-05-27T00:39:27.610354+0000","last_became_active":"2024-05-27T00:39:21.591116+0000","last_became_peered":"2024-05-27T00:39:21.591116+0000","last_unstale":"2024-05-27T00:39:27.610354+0000","last_undegraded":"2024-05-27T00:39:27.610354+0000","last_fullsized":"2024-05-27T00:39:27.610354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604797+0000","last_change":"2024-05-27T00:39:20.573838+0000","last_active":"2024-05-27T00:39:27.604797+0000","last_peered":"2024-05-27T00:39:27.604797+0000","last_clean":"2024-05-27T00:39:27.604797+0000","last_became_active":"2024-05-27T00:39:20.573515+0000","last_became_peered":"2024-05-27T00:39:20.573515+0000","last_unstale":"2024-05-27T00:39:27.604797+0000","last_undegraded":"2024-05-27T00:39:27.604797+0000","last_fullsized":"2024-05-27T00:39:27.604797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609614+0000","last_change":"2024-05-27T00:39:25.960920+0000","last_active":"2024-05-27T00:39:27.609614+0000","last_peered":"2024-05-27T00:39:27.609614+0000","last_clean":"2024-05-27T00:39:27.609614+0000","last_became_active":"2024-05-27T00:39:25.960487+0000","last_became_peered":"2024-05-27T00:39:25.960487+0000","last_unstale":"2024-05-27T00:39:27.609614+0000","last_undegraded":"2024-05-27T00:39:27.609614+0000","last_fullsized":"2024-05-27T00:39:27.609614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:59.647950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604641+0000","last_change":"2024-05-27T00:39:21.583980+0000","last_active":"2024-05-27T00:39:27.604641+0000","last_peered":"2024-05-27T00:39:27.604641+0000","last_clean":"2024-05-27T00:39:27.604641+0000","last_became_active":"2024-05-27T00:39:21.583700+0000","last_became_peered":"2024-05-27T00:39:21.583700+0000","last_unstale":"2024-05-27T00:39:27.604641+0000","last_undegraded":"2024-05-27T00:39:27.604641+0000","last_fullsized":"2024-05-27T00:39:27.604641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834010+0000","last_change":"2024-05-27T00:39:20.579198+0000","last_active":"2024-05-27T00:39:28.834010+0000","last_peered":"2024-05-27T00:39:28.834010+0000","last_clean":"2024-05-27T00:39:28.834010+0000","last_became_active":"2024-05-27T00:39:20.578902+0000","last_became_peered":"2024-05-27T00:39:20.578902+0000","last_unstale":"2024-05-27T00:39:28.834010+0000","last_undegraded":"2024-05-27T00:39:28.834010+0000","last_fullsized":"2024-05-27T00:39:28.834010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.610865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.962036+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.961152+0000","last_became_peered":"2024-05-27T00:39:25.961152+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610557+0000","last_change":"2024-05-27T00:39:21.591906+0000","last_active":"2024-05-27T00:39:27.610557+0000","last_peered":"2024-05-27T00:39:27.610557+0000","last_clean":"2024-05-27T00:39:27.610557+0000","last_became_active":"2024-05-27T00:39:21.591611+0000","last_became_peered":"2024-05-27T00:39:21.591611+0000","last_unstale":"2024-05-27T00:39:27.610557+0000","last_undegraded":"2024-05-27T00:39:27.610557+0000","last_fullsized":"2024-05-27T00:39:27.610557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604735+0000","last_change":"2024-05-27T00:39:20.569834+0000","last_active":"2024-05-27T00:39:27.604735+0000","last_peered":"2024-05-27T00:39:27.604735+0000","last_clean":"2024-05-27T00:39:27.604735+0000","last_became_active":"2024-05-27T00:39:20.569540+0000","last_became_peered":"2024-05-27T00:39:20.569540+0000","last_unstale":"2024-05-27T00:39:27.604735+0000","last_undegraded":"2024-05-27T00:39:27.604735+0000","last_fullsized":"2024-05-27T00:39:27.604735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604262+0000","last_change":"2024-05-27T00:39:25.627533+0000","last_active":"2024-05-27T00:39:27.604262+0000","last_peered":"2024-05-27T00:39:27.604262+0000","last_clean":"2024-05-27T00:39:27.604262+0000","last_became_active":"2024-05-27T00:39:25.625872+0000","last_became_peered":"2024-05-27T00:39:25.625872+0000","last_unstale":"2024-05-27T00:39:27.604262+0000","last_undegraded":"2024-05-27T00:39:27.604262+0000","last_fullsized":"2024-05-27T00:39:27.604262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834370+0000","last_change":"2024-05-27T00:39:21.598984+0000","last_active":"2024-05-27T00:39:28.834370+0000","last_peered":"2024-05-27T00:39:28.834370+0000","last_clean":"2024-05-27T00:39:28.834370+0000","last_became_active":"2024-05-27T00:39:21.594222+0000","last_became_peered":"2024-05-27T00:39:21.594222+0000","last_unstale":"2024-05-27T00:39:28.834370+0000","last_undegraded":"2024-05-27T00:39:28.834370+0000","last_fullsized":"2024-05-27T00:39:28.834370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610187+0000","last_change":"2024-05-27T00:39:20.565350+0000","last_active":"2024-05-27T00:39:27.610187+0000","last_peered":"2024-05-27T00:39:27.610187+0000","last_clean":"2024-05-27T00:39:27.610187+0000","last_became_active":"2024-05-27T00:39:20.562112+0000","last_became_peered":"2024-05-27T00:39:20.562112+0000","last_unstale":"2024-05-27T00:39:27.610187+0000","last_undegraded":"2024-05-27T00:39:27.610187+0000","last_fullsized":"2024-05-27T00:39:27.610187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604681+0000","last_change":"2024-05-27T00:39:25.624845+0000","last_active":"2024-05-27T00:39:27.604681+0000","last_peered":"2024-05-27T00:39:27.604681+0000","last_clean":"2024-05-27T00:39:27.604681+0000","last_became_active":"2024-05-27T00:39:25.624336+0000","last_became_peered":"2024-05-27T00:39:25.624336+0000","last_unstale":"2024-05-27T00:39:27.604681+0000","last_undegraded":"2024-05-27T00:39:27.604681+0000","last_fullsized":"2024-05-27T00:39:27.604681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604350+0000","last_change":"2024-05-27T00:39:21.586786+0000","last_active":"2024-05-27T00:39:27.604350+0000","last_peered":"2024-05-27T00:39:27.604350+0000","last_clean":"2024-05-27T00:39:27.604350+0000","last_became_active":"2024-05-27T00:39:21.586521+0000","last_became_peered":"2024-05-27T00:39:21.586521+0000","last_unstale":"2024-05-27T00:39:27.604350+0000","last_undegraded":"2024-05-27T00:39:27.604350+0000","last_fullsized":"2024-05-27T00:39:27.604350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573104+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.572665+0000","last_became_peered":"2024-05-27T00:39:20.572665+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605183+0000","last_change":"2024-05-27T00:39:25.625105+0000","last_active":"2024-05-27T00:39:27.605183+0000","last_peered":"2024-05-27T00:39:27.605183+0000","last_clean":"2024-05-27T00:39:27.605183+0000","last_became_active":"2024-05-27T00:39:25.624400+0000","last_became_peered":"2024-05-27T00:39:25.624400+0000","last_unstale":"2024-05-27T00:39:27.605183+0000","last_undegraded":"2024-05-27T00:39:27.605183+0000","last_fullsized":"2024-05-27T00:39:27.605183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133189+0000","last_change":"2024-05-27T00:39:21.578555+0000","last_active":"2024-05-27T00:39:28.133189+0000","last_peered":"2024-05-27T00:39:28.133189+0000","last_clean":"2024-05-27T00:39:28.133189+0000","last_became_active":"2024-05-27T00:39:21.578291+0000","last_became_peered":"2024-05-27T00:39:21.578291+0000","last_unstale":"2024-05-27T00:39:28.133189+0000","last_undegraded":"2024-05-27T00:39:28.133189+0000","last_fullsized":"2024-05-27T00:39:28.133189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:00.616291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832328+0000","last_change":"2024-05-27T00:39:20.578837+0000","last_active":"2024-05-27T00:39:28.832328+0000","last_peered":"2024-05-27T00:39:28.832328+0000","last_clean":"2024-05-27T00:39:28.832328+0000","last_became_active":"2024-05-27T00:39:20.578414+0000","last_became_peered":"2024-05-27T00:39:20.578414+0000","last_unstale":"2024-05-27T00:39:28.832328+0000","last_undegraded":"2024-05-27T00:39:28.832328+0000","last_fullsized":"2024-05-27T00:39:28.832328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133680+0000","last_change":"2024-05-27T00:39:25.962038+0000","last_active":"2024-05-27T00:39:28.133680+0000","last_peered":"2024-05-27T00:39:28.133680+0000","last_clean":"2024-05-27T00:39:28.133680+0000","last_became_active":"2024-05-27T00:39:25.961489+0000","last_became_peered":"2024-05-27T00:39:25.961489+0000","last_unstale":"2024-05-27T00:39:28.133680+0000","last_undegraded":"2024-05-27T00:39:28.133680+0000","last_fullsized":"2024-05-27T00:39:28.133680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:36.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609779+0000","last_change":"2024-05-27T00:39:21.593420+0000","last_active":"2024-05-27T00:39:27.609779+0000","last_peered":"2024-05-27T00:39:27.609779+0000","last_clean":"2024-05-27T00:39:27.609779+0000","last_became_active":"2024-05-27T00:39:21.593165+0000","last_became_peered":"2024-05-27T00:39:21.593165+0000","last_unstale":"2024-05-27T00:39:27.609779+0000","last_undegraded":"2024-05-27T00:39:27.609779+0000","last_fullsized":"2024-05-27T00:39:27.609779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610340+0000","last_change":"2024-05-27T00:39:20.573661+0000","last_active":"2024-05-27T00:39:27.610340+0000","last_peered":"2024-05-27T00:39:27.610340+0000","last_clean":"2024-05-27T00:39:27.610340+0000","last_became_active":"2024-05-27T00:39:20.572675+0000","last_became_peered":"2024-05-27T00:39:20.572675+0000","last_unstale":"2024-05-27T00:39:27.610340+0000","last_undegraded":"2024-05-27T00:39:27.610340+0000","last_fullsized":"2024-05-27T00:39:27.610340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832965+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:39:28.832965+0000","last_peered":"2024-05-27T00:39:28.832965+0000","last_clean":"2024-05-27T00:39:28.832965+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:39:28.832965+0000","last_undegraded":"2024-05-27T00:39:28.832965+0000","last_fullsized":"2024-05-27T00:39:28.832965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834052+0000","last_change":"2024-05-27T00:39:25.964404+0000","last_active":"2024-05-27T00:39:28.834052+0000","last_peered":"2024-05-27T00:39:28.834052+0000","last_clean":"2024-05-27T00:39:28.834052+0000","last_became_active":"2024-05-27T00:39:25.963955+0000","last_became_peered":"2024-05-27T00:39:25.963955+0000","last_unstale":"2024-05-27T00:39:28.834052+0000","last_undegraded":"2024-05-27T00:39:28.834052+0000","last_fullsized":"2024-05-27T00:39:28.834052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:48.698768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834325+0000","last_change":"2024-05-27T00:39:21.585811+0000","last_active":"2024-05-27T00:39:28.834325+0000","last_peered":"2024-05-27T00:39:28.834325+0000","last_clean":"2024-05-27T00:39:28.834325+0000","last_became_active":"2024-05-27T00:39:21.585466+0000","last_became_peered":"2024-05-27T00:39:21.585466+0000","last_unstale":"2024-05-27T00:39:28.834325+0000","last_undegraded":"2024-05-27T00:39:28.834325+0000","last_fullsized":"2024-05-27T00:39:28.834325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.582020+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.581718+0000","last_became_peered":"2024-05-27T00:39:20.581718+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834411+0000","last_change":"2024-05-27T00:39:25.961052+0000","last_active":"2024-05-27T00:39:28.834411+0000","last_peered":"2024-05-27T00:39:28.834411+0000","last_clean":"2024-05-27T00:39:28.834411+0000","last_became_active":"2024-05-27T00:39:25.960632+0000","last_became_peered":"2024-05-27T00:39:25.960632+0000","last_unstale":"2024-05-27T00:39:28.834411+0000","last_undegraded":"2024-05-27T00:39:28.834411+0000","last_fullsized":"2024-05-27T00:39:28.834411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610612+0000","last_change":"2024-05-27T00:39:21.587149+0000","last_active":"2024-05-27T00:39:27.610612+0000","last_peered":"2024-05-27T00:39:27.610612+0000","last_clean":"2024-05-27T00:39:27.610612+0000","last_became_active":"2024-05-27T00:39:21.585877+0000","last_became_peered":"2024-05-27T00:39:21.585877+0000","last_unstale":"2024-05-27T00:39:27.610612+0000","last_undegraded":"2024-05-27T00:39:27.610612+0000","last_fullsized":"2024-05-27T00:39:27.610612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604682+0000","last_change":"2024-05-27T00:39:20.575827+0000","last_active":"2024-05-27T00:39:27.604682+0000","last_peered":"2024-05-27T00:39:27.604682+0000","last_clean":"2024-05-27T00:39:27.604682+0000","last_became_active":"2024-05-27T00:39:20.575555+0000","last_became_peered":"2024-05-27T00:39:20.575555+0000","last_unstale":"2024-05-27T00:39:27.604682+0000","last_undegraded":"2024-05-27T00:39:27.604682+0000","last_fullsized":"2024-05-27T00:39:27.604682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133662+0000","last_change":"2024-05-27T00:39:25.626022+0000","last_active":"2024-05-27T00:39:28.133662+0000","last_peered":"2024-05-27T00:39:28.133662+0000","last_clean":"2024-05-27T00:39:28.133662+0000","last_became_active":"2024-05-27T00:39:25.625643+0000","last_became_peered":"2024-05-27T00:39:25.625643+0000","last_unstale":"2024-05-27T00:39:28.133662+0000","last_undegraded":"2024-05-27T00:39:28.133662+0000","last_fullsized":"2024-05-27T00:39:28.133662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833753+0000","last_change":"2024-05-27T00:39:21.599339+0000","last_active":"2024-05-27T00:39:28.833753+0000","last_peered":"2024-05-27T00:39:28.833753+0000","last_clean":"2024-05-27T00:39:28.833753+0000","last_became_active":"2024-05-27T00:39:21.598994+0000","last_became_peered":"2024-05-27T00:39:21.598994+0000","last_unstale":"2024-05-27T00:39:28.833753+0000","last_undegraded":"2024-05-27T00:39:28.833753+0000","last_fullsized":"2024-05-27T00:39:28.833753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:13.702562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831770+0000","last_change":"2024-05-27T00:39:20.572771+0000","last_active":"2024-05-27T00:39:28.831770+0000","last_peered":"2024-05-27T00:39:28.831770+0000","last_clean":"2024-05-27T00:39:28.831770+0000","last_became_active":"2024-05-27T00:39:20.572486+0000","last_became_peered":"2024-05-27T00:39:20.572486+0000","last_unstale":"2024-05-27T00:39:28.831770+0000","last_undegraded":"2024-05-27T00:39:28.831770+0000","last_fullsized":"2024-05-27T00:39:28.831770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609790+0000","last_change":"2024-05-27T00:39:25.962926+0000","last_active":"2024-05-27T00:39:27.609790+0000","last_peered":"2024-05-27T00:39:27.609790+0000","last_clean":"2024-05-27T00:39:27.609790+0000","last_became_active":"2024-05-27T00:39:25.962471+0000","last_became_peered":"2024-05-27T00:39:25.962471+0000","last_unstale":"2024-05-27T00:39:27.609790+0000","last_undegraded":"2024-05-27T00:39:27.609790+0000","last_fullsized":"2024-05-27T00:39:27.609790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833146+0000","last_change":"2024-05-27T00:39:21.596571+0000","last_active":"2024-05-27T00:39:28.833146+0000","last_peered":"2024-05-27T00:39:28.833146+0000","last_clean":"2024-05-27T00:39:28.833146+0000","last_became_active":"2024-05-27T00:39:21.596278+0000","last_became_peered":"2024-05-27T00:39:21.596278+0000","last_unstale":"2024-05-27T00:39:28.833146+0000","last_undegraded":"2024-05-27T00:39:28.833146+0000","last_fullsized":"2024-05-27T00:39:28.833146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:42.881822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832599+0000","last_change":"2024-05-27T00:39:20.575461+0000","last_active":"2024-05-27T00:39:28.832599+0000","last_peered":"2024-05-27T00:39:28.832599+0000","last_clean":"2024-05-27T00:39:28.832599+0000","last_became_active":"2024-05-27T00:39:20.575168+0000","last_became_peered":"2024-05-27T00:39:20.575168+0000","last_unstale":"2024-05-27T00:39:28.832599+0000","last_undegraded":"2024-05-27T00:39:28.832599+0000","last_fullsized":"2024-05-27T00:39:28.832599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:04:51.312690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832581+0000","last_change":"2024-05-27T00:39:25.627556+0000","last_active":"2024-05-27T00:39:28.832581+0000","last_peered":"2024-05-27T00:39:28.832581+0000","last_clean":"2024-05-27T00:39:28.832581+0000","last_became_active":"2024-05-27T00:39:25.627169+0000","last_became_peered":"2024-05-27T00:39:25.627169+0000","last_unstale":"2024-05-27T00:39:28.832581+0000","last_undegraded":"2024-05-27T00:39:28.832581+0000","last_fullsized":"2024-05-27T00:39:28.832581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832699+0000","last_change":"2024-05-27T00:39:21.596905+0000","last_active":"2024-05-27T00:39:28.832699+0000","last_peered":"2024-05-27T00:39:28.832699+0000","last_clean":"2024-05-27T00:39:28.832699+0000","last_became_active":"2024-05-27T00:39:21.596586+0000","last_became_peered":"2024-05-27T00:39:21.596586+0000","last_unstale":"2024-05-27T00:39:28.832699+0000","last_undegraded":"2024-05-27T00:39:28.832699+0000","last_fullsized":"2024-05-27T00:39:28.832699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573588+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.573302+0000","last_became_peered":"2024-05-27T00:39:20.573302+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.620609+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.620185+0000","last_became_peered":"2024-05-27T00:39:25.620185+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835033+0000","last_change":"2024-05-27T00:39:21.593089+0000","last_active":"2024-05-27T00:39:28.835033+0000","last_peered":"2024-05-27T00:39:28.835033+0000","last_clean":"2024-05-27T00:39:28.835033+0000","last_became_active":"2024-05-27T00:39:21.592034+0000","last_became_peered":"2024-05-27T00:39:21.592034+0000","last_unstale":"2024-05-27T00:39:28.835033+0000","last_undegraded":"2024-05-27T00:39:28.835033+0000","last_fullsized":"2024-05-27T00:39:28.835033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834807+0000","last_change":"2024-05-27T00:39:20.576289+0000","last_active":"2024-05-27T00:39:28.834807+0000","last_peered":"2024-05-27T00:39:28.834807+0000","last_clean":"2024-05-27T00:39:28.834807+0000","last_became_active":"2024-05-27T00:39:20.576008+0000","last_became_peered":"2024-05-27T00:39:20.576008+0000","last_unstale":"2024-05-27T00:39:28.834807+0000","last_undegraded":"2024-05-27T00:39:28.834807+0000","last_fullsized":"2024-05-27T00:39:28.834807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610207+0000","last_change":"2024-05-27T00:39:25.964215+0000","last_active":"2024-05-27T00:39:27.610207+0000","last_peered":"2024-05-27T00:39:27.610207+0000","last_clean":"2024-05-27T00:39:27.610207+0000","last_became_active":"2024-05-27T00:39:25.963695+0000","last_became_peered":"2024-05-27T00:39:25.963695+0000","last_unstale":"2024-05-27T00:39:27.610207+0000","last_undegraded":"2024-05-27T00:39:27.610207+0000","last_fullsized":"2024-05-27T00:39:27.610207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:00:27.182513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833721+0000","last_change":"2024-05-27T00:39:21.599920+0000","last_active":"2024-05-27T00:39:28.833721+0000","last_peered":"2024-05-27T00:39:28.833721+0000","last_clean":"2024-05-27T00:39:28.833721+0000","last_became_active":"2024-05-27T00:39:21.597737+0000","last_became_peered":"2024-05-27T00:39:21.597737+0000","last_unstale":"2024-05-27T00:39:28.833721+0000","last_undegraded":"2024-05-27T00:39:28.833721+0000","last_fullsized":"2024-05-27T00:39:28.833721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:38.014686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610101+0000","last_change":"2024-05-27T00:39:20.571178+0000","last_active":"2024-05-27T00:39:27.610101+0000","last_peered":"2024-05-27T00:39:27.610101+0000","last_clean":"2024-05-27T00:39:27.610101+0000","last_became_active":"2024-05-27T00:39:20.570884+0000","last_became_peered":"2024-05-27T00:39:20.570884+0000","last_unstale":"2024-05-27T00:39:27.610101+0000","last_undegraded":"2024-05-27T00:39:27.610101+0000","last_fullsized":"2024-05-27T00:39:27.610101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610204+0000","last_change":"2024-05-27T00:39:25.961014+0000","last_active":"2024-05-27T00:39:27.610204+0000","last_peered":"2024-05-27T00:39:27.610204+0000","last_clean":"2024-05-27T00:39:27.610204+0000","last_became_active":"2024-05-27T00:39:25.960630+0000","last_became_peered":"2024-05-27T00:39:25.960630+0000","last_unstale":"2024-05-27T00:39:27.610204+0000","last_undegraded":"2024-05-27T00:39:27.610204+0000","last_fullsized":"2024-05-27T00:39:27.610204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:53:07.966829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133320+0000","last_change":"2024-05-27T00:39:21.583015+0000","last_active":"2024-05-27T00:39:28.133320+0000","last_peered":"2024-05-27T00:39:28.133320+0000","last_clean":"2024-05-27T00:39:28.133320+0000","last_became_active":"2024-05-27T00:39:21.582722+0000","last_became_peered":"2024-05-27T00:39:21.582722+0000","last_unstale":"2024-05-27T00:39:28.133320+0000","last_undegraded":"2024-05-27T00:39:28.133320+0000","last_fullsized":"2024-05-27T00:39:28.133320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:04.988100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133718+0000","last_change":"2024-05-27T00:39:20.575435+0000","last_active":"2024-05-27T00:39:28.133718+0000","last_peered":"2024-05-27T00:39:28.133718+0000","last_clean":"2024-05-27T00:39:28.133718+0000","last_became_active":"2024-05-27T00:39:20.575144+0000","last_became_peered":"2024-05-27T00:39:20.575144+0000","last_unstale":"2024-05-27T00:39:28.133718+0000","last_undegraded":"2024-05-27T00:39:28.133718+0000","last_fullsized":"2024-05-27T00:39:28.133718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:25:38.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610849+0000","last_change":"2024-05-27T00:39:25.961559+0000","last_active":"2024-05-27T00:39:27.610849+0000","last_peered":"2024-05-27T00:39:27.610849+0000","last_clean":"2024-05-27T00:39:27.610849+0000","last_became_active":"2024-05-27T00:39:25.961155+0000","last_became_peered":"2024-05-27T00:39:25.961155+0000","last_unstale":"2024-05-27T00:39:27.610849+0000","last_undegraded":"2024-05-27T00:39:27.610849+0000","last_fullsized":"2024-05-27T00:39:27.610849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834212+0000","last_change":"2024-05-27T00:39:21.593738+0000","last_active":"2024-05-27T00:39:28.834212+0000","last_peered":"2024-05-27T00:39:28.834212+0000","last_clean":"2024-05-27T00:39:28.834212+0000","last_became_active":"2024-05-27T00:39:21.591298+0000","last_became_peered":"2024-05-27T00:39:21.591298+0000","last_unstale":"2024-05-27T00:39:28.834212+0000","last_undegraded":"2024-05-27T00:39:28.834212+0000","last_fullsized":"2024-05-27T00:39:28.834212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:30.527235+0000","last_change":"2024-05-27T00:39:20.577937+0000","last_active":"2024-05-27T00:39:30.527235+0000","last_peered":"2024-05-27T00:39:30.527235+0000","last_clean":"2024-05-27T00:39:30.527235+0000","last_became_active":"2024-05-27T00:39:20.577630+0000","last_became_peered":"2024-05-27T00:39:20.577630+0000","last_unstale":"2024-05-27T00:39:30.527235+0000","last_undegraded":"2024-05-27T00:39:30.527235+0000","last_fullsized":"2024-05-27T00:39:30.527235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604388+0000","last_change":"2024-05-27T00:39:25.620344+0000","last_active":"2024-05-27T00:39:27.604388+0000","last_peered":"2024-05-27T00:39:27.604388+0000","last_clean":"2024-05-27T00:39:27.604388+0000","last_became_active":"2024-05-27T00:39:25.618090+0000","last_became_peered":"2024-05-27T00:39:25.618090+0000","last_unstale":"2024-05-27T00:39:27.604388+0000","last_undegraded":"2024-05-27T00:39:27.604388+0000","last_fullsized":"2024-05-27T00:39:27.604388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834698+0000","last_change":"2024-05-27T00:39:21.593265+0000","last_active":"2024-05-27T00:39:28.834698+0000","last_peered":"2024-05-27T00:39:28.834698+0000","last_clean":"2024-05-27T00:39:28.834698+0000","last_became_active":"2024-05-27T00:39:21.592984+0000","last_became_peered":"2024-05-27T00:39:21.592984+0000","last_unstale":"2024-05-27T00:39:28.834698+0000","last_undegraded":"2024-05-27T00:39:28.834698+0000","last_fullsized":"2024-05-27T00:39:28.834698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:06.451979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604736+0000","last_change":"2024-05-27T00:39:20.570519+0000","last_active":"2024-05-27T00:39:27.604736+0000","last_peered":"2024-05-27T00:39:27.604736+0000","last_clean":"2024-05-27T00:39:27.604736+0000","last_became_active":"2024-05-27T00:39:20.570230+0000","last_became_peered":"2024-05-27T00:39:20.570230+0000","last_unstale":"2024-05-27T00:39:27.604736+0000","last_undegraded":"2024-05-27T00:39:27.604736+0000","last_fullsized":"2024-05-27T00:39:27.604736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834196+0000","last_change":"2024-05-27T00:39:25.628554+0000","last_active":"2024-05-27T00:39:28.834196+0000","last_peered":"2024-05-27T00:39:28.834196+0000","last_clean":"2024-05-27T00:39:28.834196+0000","last_became_active":"2024-05-27T00:39:25.627971+0000","last_became_peered":"2024-05-27T00:39:25.627971+0000","last_unstale":"2024-05-27T00:39:28.834196+0000","last_undegraded":"2024-05-27T00:39:28.834196+0000","last_fullsized":"2024-05-27T00:39:28.834196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133800+0000","last_change":"2024-05-27T00:39:21.576653+0000","last_active":"2024-05-27T00:39:28.133800+0000","last_peered":"2024-05-27T00:39:28.133800+0000","last_clean":"2024-05-27T00:39:28.133800+0000","last_became_active":"2024-05-27T00:39:21.576383+0000","last_became_peered":"2024-05-27T00:39:21.576383+0000","last_unstale":"2024-05-27T00:39:28.133800+0000","last_undegraded":"2024-05-27T00:39:28.133800+0000","last_fullsized":"2024-05-27T00:39:28.133800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610702+0000","last_change":"2024-05-27T00:39:20.575867+0000","last_active":"2024-05-27T00:39:27.610702+0000","last_peered":"2024-05-27T00:39:27.610702+0000","last_clean":"2024-05-27T00:39:27.610702+0000","last_became_active":"2024-05-27T00:39:20.575557+0000","last_became_peered":"2024-05-27T00:39:20.575557+0000","last_unstale":"2024-05-27T00:39:27.610702+0000","last_undegraded":"2024-05-27T00:39:27.610702+0000","last_fullsized":"2024-05-27T00:39:27.610702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609795+0000","last_change":"2024-05-27T00:39:25.627020+0000","last_active":"2024-05-27T00:39:27.609795+0000","last_peered":"2024-05-27T00:39:27.609795+0000","last_clean":"2024-05-27T00:39:27.609795+0000","last_became_active":"2024-05-27T00:39:25.626629+0000","last_became_peered":"2024-05-27T00:39:25.626629+0000","last_unstale":"2024-05-27T00:39:27.609795+0000","last_undegraded":"2024-05-27T00:39:27.609795+0000","last_fullsized":"2024-05-27T00:39:27.609795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:57.470204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610617+0000","last_change":"2024-05-27T00:39:21.584614+0000","last_active":"2024-05-27T00:39:27.610617+0000","last_peered":"2024-05-27T00:39:27.610617+0000","last_clean":"2024-05-27T00:39:27.610617+0000","last_became_active":"2024-05-27T00:39:21.584349+0000","last_became_peered":"2024-05-27T00:39:21.584349+0000","last_unstale":"2024-05-27T00:39:27.610617+0000","last_undegraded":"2024-05-27T00:39:27.610617+0000","last_fullsized":"2024-05-27T00:39:27.610617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834142+0000","last_change":"2024-05-27T00:39:20.575901+0000","last_active":"2024-05-27T00:39:28.834142+0000","last_peered":"2024-05-27T00:39:28.834142+0000","last_clean":"2024-05-27T00:39:28.834142+0000","last_became_active":"2024-05-27T00:39:20.575521+0000","last_became_peered":"2024-05-27T00:39:20.575521+0000","last_unstale":"2024-05-27T00:39:28.834142+0000","last_undegraded":"2024-05-27T00:39:28.834142+0000","last_fullsized":"2024-05-27T00:39:28.834142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609706+0000","last_change":"2024-05-27T00:39:25.961740+0000","last_active":"2024-05-27T00:39:27.609706+0000","last_peered":"2024-05-27T00:39:27.609706+0000","last_clean":"2024-05-27T00:39:27.609706+0000","last_became_active":"2024-05-27T00:39:25.960524+0000","last_became_peered":"2024-05-27T00:39:25.960524+0000","last_unstale":"2024-05-27T00:39:27.609706+0000","last_undegraded":"2024-05-27T00:39:27.609706+0000","last_fullsized":"2024-05-27T00:39:27.609706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:38:40.065860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133581+0000","last_change":"2024-05-27T00:39:25.961849+0000","last_active":"2024-05-27T00:39:28.133581+0000","last_peered":"2024-05-27T00:39:28.133581+0000","last_clean":"2024-05-27T00:39:28.133581+0000","last_became_active":"2024-05-27T00:39:25.960998+0000","last_became_peered":"2024-05-27T00:39:25.960998+0000","last_unstale":"2024-05-27T00:39:28.133581+0000","last_undegraded":"2024-05-27T00:39:28.133581+0000","last_fullsized":"2024-05-27T00:39:28.133581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604056+0000","last_change":"2024-05-27T00:39:21.590501+0000","last_active":"2024-05-27T00:39:27.604056+0000","last_peered":"2024-05-27T00:39:27.604056+0000","last_clean":"2024-05-27T00:39:27.604056+0000","last_became_active":"2024-05-27T00:39:21.590246+0000","last_became_peered":"2024-05-27T00:39:21.590246+0000","last_unstale":"2024-05-27T00:39:27.604056+0000","last_undegraded":"2024-05-27T00:39:27.604056+0000","last_fullsized":"2024-05-27T00:39:27.604056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609915+0000","last_change":"2024-05-27T00:39:20.574994+0000","last_active":"2024-05-27T00:39:27.609915+0000","last_peered":"2024-05-27T00:39:27.609915+0000","last_clean":"2024-05-27T00:39:27.609915+0000","last_became_active":"2024-05-27T00:39:20.574713+0000","last_became_peered":"2024-05-27T00:39:20.574713+0000","last_unstale":"2024-05-27T00:39:27.609915+0000","last_undegraded":"2024-05-27T00:39:27.609915+0000","last_fullsized":"2024-05-27T00:39:27.609915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835197+0000","last_change":"2024-05-27T00:39:25.629723+0000","last_active":"2024-05-27T00:39:28.835197+0000","last_peered":"2024-05-27T00:39:28.835197+0000","last_clean":"2024-05-27T00:39:28.835197+0000","last_became_active":"2024-05-27T00:39:25.628168+0000","last_became_peered":"2024-05-27T00:39:25.628168+0000","last_unstale":"2024-05-27T00:39:28.835197+0000","last_undegraded":"2024-05-27T00:39:28.835197+0000","last_fullsized":"2024-05-27T00:39:28.835197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609695+0000","last_change":"2024-05-27T00:39:21.591614+0000","last_active":"2024-05-27T00:39:27.609695+0000","last_peered":"2024-05-27T00:39:27.609695+0000","last_clean":"2024-05-27T00:39:27.609695+0000","last_became_active":"2024-05-27T00:39:21.591359+0000","last_became_peered":"2024-05-27T00:39:21.591359+0000","last_unstale":"2024-05-27T00:39:27.609695+0000","last_undegraded":"2024-05-27T00:39:27.609695+0000","last_fullsized":"2024-05-27T00:39:27.609695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610097+0000","last_change":"2024-05-27T00:39:20.575532+0000","last_active":"2024-05-27T00:39:27.610097+0000","last_peered":"2024-05-27T00:39:27.610097+0000","last_clean":"2024-05-27T00:39:27.610097+0000","last_became_active":"2024-05-27T00:39:20.575201+0000","last_became_peered":"2024-05-27T00:39:20.575201+0000","last_unstale":"2024-05-27T00:39:27.610097+0000","last_undegraded":"2024-05-27T00:39:27.610097+0000","last_fullsized":"2024-05-27T00:39:27.610097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834640+0000","last_change":"2024-05-27T00:39:20.577031+0000","last_active":"2024-05-27T00:39:28.834640+0000","last_peered":"2024-05-27T00:39:28.834640+0000","last_clean":"2024-05-27T00:39:28.834640+0000","last_became_active":"2024-05-27T00:39:20.576619+0000","last_became_peered":"2024-05-27T00:39:20.576619+0000","last_unstale":"2024-05-27T00:39:28.834640+0000","last_undegraded":"2024-05-27T00:39:28.834640+0000","last_fullsized":"2024-05-27T00:39:28.834640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133684+0000","last_change":"2024-05-27T00:39:21.580671+0000","last_active":"2024-05-27T00:39:28.133684+0000","last_peered":"2024-05-27T00:39:28.133684+0000","last_clean":"2024-05-27T00:39:28.133684+0000","last_became_active":"2024-05-27T00:39:21.580434+0000","last_became_peered":"2024-05-27T00:39:21.580434+0000","last_unstale":"2024-05-27T00:39:28.133684+0000","last_undegraded":"2024-05-27T00:39:28.133684+0000","last_fullsized":"2024-05-27T00:39:28.133684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832153+0000","last_change":"2024-05-27T00:39:25.624658+0000","last_active":"2024-05-27T00:39:28.832153+0000","last_peered":"2024-05-27T00:39:28.832153+0000","last_clean":"2024-05-27T00:39:28.832153+0000","last_became_active":"2024-05-27T00:39:25.624289+0000","last_became_peered":"2024-05-27T00:39:25.624289+0000","last_unstale":"2024-05-27T00:39:28.832153+0000","last_undegraded":"2024-05-27T00:39:28.832153+0000","last_fullsized":"2024-05-27T00:39:28.832153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:22.409161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609979+0000","last_change":"2024-05-27T00:39:20.572852+0000","last_active":"2024-05-27T00:39:27.609979+0000","last_peered":"2024-05-27T00:39:27.609979+0000","last_clean":"2024-05-27T00:39:27.609979+0000","last_became_active":"2024-05-27T00:39:20.572562+0000","last_became_peered":"2024-05-27T00:39:20.572562+0000","last_unstale":"2024-05-27T00:39:27.609979+0000","last_undegraded":"2024-05-27T00:39:27.609979+0000","last_fullsized":"2024-05-27T00:39:27.609979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833202+0000","last_change":"2024-05-27T00:39:21.587846+0000","last_active":"2024-05-27T00:39:28.833202+0000","last_peered":"2024-05-27T00:39:28.833202+0000","last_clean":"2024-05-27T00:39:28.833202+0000","last_became_active":"2024-05-27T00:39:21.587579+0000","last_became_peered":"2024-05-27T00:39:21.587579+0000","last_unstale":"2024-05-27T00:39:28.833202+0000","last_undegraded":"2024-05-27T00:39:28.833202+0000","last_fullsized":"2024-05-27T00:39:28.833202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:47.382970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610160+0000","last_change":"2024-05-27T00:39:25.628905+0000","last_active":"2024-05-27T00:39:27.610160+0000","last_peered":"2024-05-27T00:39:27.610160+0000","last_clean":"2024-05-27T00:39:27.610160+0000","last_became_active":"2024-05-27T00:39:25.628229+0000","last_became_peered":"2024-05-27T00:39:25.628229+0000","last_unstale":"2024-05-27T00:39:27.610160+0000","last_undegraded":"2024-05-27T00:39:27.610160+0000","last_fullsized":"2024-05-27T00:39:27.610160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:40:24.502057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834905+0000","last_change":"2024-05-27T00:39:21.593542+0000","last_active":"2024-05-27T00:39:28.834905+0000","last_peered":"2024-05-27T00:39:28.834905+0000","last_clean":"2024-05-27T00:39:28.834905+0000","last_became_active":"2024-05-27T00:39:21.592466+0000","last_became_peered":"2024-05-27T00:39:21.592466+0000","last_unstale":"2024-05-27T00:39:28.834905+0000","last_undegraded":"2024-05-27T00:39:28.834905+0000","last_fullsized":"2024-05-27T00:39:28.834905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831986+0000","last_change":"2024-05-27T00:39:21.586207+0000","last_active":"2024-05-27T00:39:28.831986+0000","last_peered":"2024-05-27T00:39:28.831986+0000","last_clean":"2024-05-27T00:39:28.831986+0000","last_became_active":"2024-05-27T00:39:21.585899+0000","last_became_peered":"2024-05-27T00:39:21.585899+0000","last_unstale":"2024-05-27T00:39:28.831986+0000","last_undegraded":"2024-05-27T00:39:28.831986+0000","last_fullsized":"2024-05-27T00:39:28.831986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610077+0000","last_change":"2024-05-27T00:39:21.589304+0000","last_active":"2024-05-27T00:39:27.610077+0000","last_peered":"2024-05-27T00:39:27.610077+0000","last_clean":"2024-05-27T00:39:27.610077+0000","last_became_active":"2024-05-27T00:39:21.588424+0000","last_became_peered":"2024-05-27T00:39:21.588424+0000","last_unstale":"2024-05-27T00:39:27.610077+0000","last_undegraded":"2024-05-27T00:39:27.610077+0000","last_fullsized":"2024-05-27T00:39:27.610077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133594+0000","last_change":"2024-05-27T00:39:21.587527+0000","last_active":"2024-05-27T00:39:28.133594+0000","last_peered":"2024-05-27T00:39:28.133594+0000","last_clean":"2024-05-27T00:39:28.133594+0000","last_became_active":"2024-05-27T00:39:21.587271+0000","last_became_peered":"2024-05-27T00:39:21.587271+0000","last_unstale":"2024-05-27T00:39:28.133594+0000","last_undegraded":"2024-05-27T00:39:28.133594+0000","last_fullsized":"2024-05-27T00:39:28.133594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610312+0000","last_change":"2024-05-27T00:39:21.600567+0000","last_active":"2024-05-27T00:39:27.610312+0000","last_peered":"2024-05-27T00:39:27.610312+0000","last_clean":"2024-05-27T00:39:27.610312+0000","last_became_active":"2024-05-27T00:39:21.600281+0000","last_became_peered":"2024-05-27T00:39:21.600281+0000","last_unstale":"2024-05-27T00:39:27.610312+0000","last_undegraded":"2024-05-27T00:39:27.610312+0000","last_fullsized":"2024-05-27T00:39:27.610312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604571+0000","last_change":"2024-05-27T00:39:21.591200+0000","last_active":"2024-05-27T00:39:27.604571+0000","last_peered":"2024-05-27T00:39:27.604571+0000","last_clean":"2024-05-27T00:39:27.604571+0000","last_became_active":"2024-05-27T00:39:21.590918+0000","last_became_peered":"2024-05-27T00:39:21.590918+0000","last_unstale":"2024-05-27T00:39:27.604571+0000","last_undegraded":"2024-05-27T00:39:27.604571+0000","last_fullsized":"2024-05-27T00:39:27.604571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835139+0000","last_change":"2024-05-27T00:39:21.593373+0000","last_active":"2024-05-27T00:39:28.835139+0000","last_peered":"2024-05-27T00:39:28.835139+0000","last_clean":"2024-05-27T00:39:28.835139+0000","last_became_active":"2024-05-27T00:39:21.590646+0000","last_became_peered":"2024-05-27T00:39:21.590646+0000","last_unstale":"2024-05-27T00:39:28.835139+0000","last_undegraded":"2024-05-27T00:39:28.835139+0000","last_fullsized":"2024-05-27T00:39:28.835139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610260+0000","last_change":"2024-05-27T00:39:21.600531+0000","last_active":"2024-05-27T00:39:27.610260+0000","last_peered":"2024-05-27T00:39:27.610260+0000","last_clean":"2024-05-27T00:39:27.610260+0000","last_became_active":"2024-05-27T00:39:21.600236+0000","last_became_peered":"2024-05-27T00:39:21.600236+0000","last_unstale":"2024-05-27T00:39:27.610260+0000","last_undegraded":"2024-05-27T00:39:27.610260+0000","last_fullsized":"2024-05-27T00:39:27.610260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609848+0000","last_change":"2024-05-27T00:39:21.588532+0000","last_active":"2024-05-27T00:39:27.609848+0000","last_peered":"2024-05-27T00:39:27.609848+0000","last_clean":"2024-05-27T00:39:27.609848+0000","last_became_active":"2024-05-27T00:39:21.587475+0000","last_became_peered":"2024-05-27T00:39:21.587475+0000","last_unstale":"2024-05-27T00:39:27.609848+0000","last_undegraded":"2024-05-27T00:39:27.609848+0000","last_fullsized":"2024-05-27T00:39:27.609848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609903+0000","last_change":"2024-05-27T00:39:21.601094+0000","last_active":"2024-05-27T00:39:27.609903+0000","last_peered":"2024-05-27T00:39:27.609903+0000","last_clean":"2024-05-27T00:39:27.609903+0000","last_became_active":"2024-05-27T00:39:21.600345+0000","last_became_peered":"2024-05-27T00:39:21.600345+0000","last_unstale":"2024-05-27T00:39:27.609903+0000","last_undegraded":"2024-05-27T00:39:27.609903+0000","last_fullsized":"2024-05-27T00:39:27.609903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609931+0000","last_change":"2024-05-27T00:39:21.583984+0000","last_active":"2024-05-27T00:39:27.609931+0000","last_peered":"2024-05-27T00:39:27.609931+0000","last_clean":"2024-05-27T00:39:27.609931+0000","last_became_active":"2024-05-27T00:39:21.583601+0000","last_became_peered":"2024-05-27T00:39:21.583601+0000","last_unstale":"2024-05-27T00:39:27.609931+0000","last_undegraded":"2024-05-27T00:39:27.609931+0000","last_fullsized":"2024-05-27T00:39:27.609931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133437+0000","last_change":"2024-05-27T00:39:21.583713+0000","last_active":"2024-05-27T00:39:28.133437+0000","last_peered":"2024-05-27T00:39:28.133437+0000","last_clean":"2024-05-27T00:39:28.133437+0000","last_became_active":"2024-05-27T00:39:21.583367+0000","last_became_peered":"2024-05-27T00:39:21.583367+0000","last_unstale":"2024-05-27T00:39:28.133437+0000","last_undegraded":"2024-05-27T00:39:28.133437+0000","last_fullsized":"2024-05-27T00:39:28.133437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834031+0000","last_change":"2024-05-27T00:39:21.597145+0000","last_active":"2024-05-27T00:39:28.834031+0000","last_peered":"2024-05-27T00:39:28.834031+0000","last_clean":"2024-05-27T00:39:28.834031+0000","last_became_active":"2024-05-27T00:39:21.596737+0000","last_became_peered":"2024-05-27T00:39:21.596737+0000","last_unstale":"2024-05-27T00:39:28.834031+0000","last_undegraded":"2024-05-27T00:39:28.834031+0000","last_fullsized":"2024-05-27T00:39:28.834031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834671+0000","last_change":"2024-05-27T00:39:21.586729+0000","last_active":"2024-05-27T00:39:28.834671+0000","last_peered":"2024-05-27T00:39:28.834671+0000","last_clean":"2024-05-27T00:39:28.834671+0000","last_became_active":"2024-05-27T00:39:21.586428+0000","last_became_peered":"2024-05-27T00:39:21.586428+0000","last_unstale":"2024-05-27T00:39:28.834671+0000","last_undegraded":"2024-05-27T00:39:28.834671+0000","last_fullsized":"2024-05-27T00:39:28.834671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832018+0000","last_change":"2024-05-27T00:39:21.591955+0000","last_active":"2024-05-27T00:39:28.832018+0000","last_peered":"2024-05-27T00:39:28.832018+0000","last_clean":"2024-05-27T00:39:28.832018+0000","last_became_active":"2024-05-27T00:39:21.591645+0000","last_became_peered":"2024-05-27T00:39:21.591645+0000","last_unstale":"2024-05-27T00:39:28.832018+0000","last_undegraded":"2024-05-27T00:39:28.832018+0000","last_fullsized":"2024-05-27T00:39:28.832018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610938+0000","last_change":"2024-05-27T00:39:21.582309+0000","last_active":"2024-05-27T00:39:27.610938+0000","last_peered":"2024-05-27T00:39:27.610938+0000","last_clean":"2024-05-27T00:39:27.610938+0000","last_became_active":"2024-05-27T00:39:21.582029+0000","last_became_peered":"2024-05-27T00:39:21.582029+0000","last_unstale":"2024-05-27T00:39:27.610938+0000","last_undegraded":"2024-05-27T00:39:27.610938+0000","last_fullsized":"2024-05-27T00:39:27.610938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609792+0000","last_change":"2024-05-27T00:39:21.583269+0000","last_active":"2024-05-27T00:39:27.609792+0000","last_peered":"2024-05-27T00:39:27.609792+0000","last_clean":"2024-05-27T00:39:27.609792+0000","last_became_active":"2024-05-27T00:39:21.582827+0000","last_became_peered":"2024-05-27T00:39:21.582827+0000","last_unstale":"2024-05-27T00:39:27.609792+0000","last_undegraded":"2024-05-27T00:39:27.609792+0000","last_fullsized":"2024-05-27T00:39:27.609792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604091+0000","last_change":"2024-05-27T00:39:21.600540+0000","last_active":"2024-05-27T00:39:27.604091+0000","last_peered":"2024-05-27T00:39:27.604091+0000","last_clean":"2024-05-27T00:39:27.604091+0000","last_became_active":"2024-05-27T00:39:21.600270+0000","last_became_peered":"2024-05-27T00:39:21.600270+0000","last_unstale":"2024-05-27T00:39:27.604091+0000","last_undegraded":"2024-05-27T00:39:27.604091+0000","last_fullsized":"2024-05-27T00:39:27.604091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133315+0000","last_change":"2024-05-27T00:39:21.587064+0000","last_active":"2024-05-27T00:39:28.133315+0000","last_peered":"2024-05-27T00:39:28.133315+0000","last_clean":"2024-05-27T00:39:28.133315+0000","last_became_active":"2024-05-27T00:39:21.586759+0000","last_became_peered":"2024-05-27T00:39:21.586759+0000","last_unstale":"2024-05-27T00:39:28.133315+0000","last_undegraded":"2024-05-27T00:39:28.133315+0000","last_fullsized":"2024-05-27T00:39:28.133315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:18.374156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609806+0000","last_change":"2024-05-27T00:39:21.594918+0000","last_active":"2024-05-27T00:39:27.609806+0000","last_peered":"2024-05-27T00:39:27.609806+0000","last_clean":"2024-05-27T00:39:27.609806+0000","last_became_active":"2024-05-27T00:39:21.594670+0000","last_became_peered":"2024-05-27T00:39:21.594670+0000","last_unstale":"2024-05-27T00:39:27.609806+0000","last_undegraded":"2024-05-27T00:39:27.609806+0000","last_fullsized":"2024-05-27T00:39:27.609806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834590+0000","last_change":"2024-05-27T00:39:21.600525+0000","last_active":"2024-05-27T00:39:28.834590+0000","last_peered":"2024-05-27T00:39:28.834590+0000","last_clean":"2024-05-27T00:39:28.834590+0000","last_became_active":"2024-05-27T00:39:21.600012+0000","last_became_peered":"2024-05-27T00:39:21.600012+0000","last_unstale":"2024-05-27T00:39:28.834590+0000","last_undegraded":"2024-05-27T00:39:28.834590+0000","last_fullsized":"2024-05-27T00:39:28.834590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834173+0000","last_change":"2024-05-27T00:39:21.596158+0000","last_active":"2024-05-27T00:39:28.834173+0000","last_peered":"2024-05-27T00:39:28.834173+0000","last_clean":"2024-05-27T00:39:28.834173+0000","last_became_active":"2024-05-27T00:39:21.595655+0000","last_became_peered":"2024-05-27T00:39:21.595655+0000","last_unstale":"2024-05-27T00:39:28.834173+0000","last_undegraded":"2024-05-27T00:39:28.834173+0000","last_fullsized":"2024-05-27T00:39:28.834173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835071+0000","last_change":"2024-05-27T00:39:21.594957+0000","last_active":"2024-05-27T00:39:28.835071+0000","last_peered":"2024-05-27T00:39:28.835071+0000","last_clean":"2024-05-27T00:39:28.835071+0000","last_became_active":"2024-05-27T00:39:21.594682+0000","last_became_peered":"2024-05-27T00:39:21.594682+0000","last_unstale":"2024-05-27T00:39:28.835071+0000","last_undegraded":"2024-05-27T00:39:28.835071+0000","last_fullsized":"2024-05-27T00:39:28.835071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833653+0000","last_change":"2024-05-27T00:39:21.589852+0000","last_active":"2024-05-27T00:39:28.833653+0000","last_peered":"2024-05-27T00:39:28.833653+0000","last_clean":"2024-05-27T00:39:28.833653+0000","last_became_active":"2024-05-27T00:39:21.589514+0000","last_became_peered":"2024-05-27T00:39:21.589514+0000","last_unstale":"2024-05-27T00:39:28.833653+0000","last_undegraded":"2024-05-27T00:39:28.833653+0000","last_fullsized":"2024-05-27T00:39:28.833653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:08.796084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609583+0000","last_change":"2024-05-27T00:39:21.586007+0000","last_active":"2024-05-27T00:39:27.609583+0000","last_peered":"2024-05-27T00:39:27.609583+0000","last_clean":"2024-05-27T00:39:27.609583+0000","last_became_active":"2024-05-27T00:39:21.585708+0000","last_became_peered":"2024-05-27T00:39:21.585708+0000","last_unstale":"2024-05-27T00:39:27.609583+0000","last_undegraded":"2024-05-27T00:39:27.609583+0000","last_fullsized":"2024-05-27T00:39:27.609583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832658+0000","last_change":"2024-05-27T00:39:21.596377+0000","last_active":"2024-05-27T00:39:28.832658+0000","last_peered":"2024-05-27T00:39:28.832658+0000","last_clean":"2024-05-27T00:39:28.832658+0000","last_became_active":"2024-05-27T00:39:21.596055+0000","last_became_peered":"2024-05-27T00:39:21.596055+0000","last_unstale":"2024-05-27T00:39:28.832658+0000","last_undegraded":"2024-05-27T00:39:28.832658+0000","last_fullsized":"2024-05-27T00:39:28.832658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609929+0000","last_change":"2024-05-27T00:39:21.583846+0000","last_active":"2024-05-27T00:39:27.609929+0000","last_peered":"2024-05-27T00:39:27.609929+0000","last_clean":"2024-05-27T00:39:27.609929+0000","last_became_active":"2024-05-27T00:39:21.583284+0000","last_became_peered":"2024-05-27T00:39:21.583284+0000","last_unstale":"2024-05-27T00:39:27.609929+0000","last_undegraded":"2024-05-27T00:39:27.609929+0000","last_fullsized":"2024-05-27T00:39:27.609929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833536+0000","last_change":"2024-05-27T00:39:21.586873+0000","last_active":"2024-05-27T00:39:28.833536+0000","last_peered":"2024-05-27T00:39:28.833536+0000","last_clean":"2024-05-27T00:39:28.833536+0000","last_became_active":"2024-05-27T00:39:21.586593+0000","last_became_peered":"2024-05-27T00:39:21.586593+0000","last_unstale":"2024-05-27T00:39:28.833536+0000","last_undegraded":"2024-05-27T00:39:28.833536+0000","last_fullsized":"2024-05-27T00:39:28.833536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610399+0000","last_change":"2024-05-27T00:39:21.592963+0000","last_active":"2024-05-27T00:39:27.610399+0000","last_peered":"2024-05-27T00:39:27.610399+0000","last_clean":"2024-05-27T00:39:27.610399+0000","last_became_active":"2024-05-27T00:39:21.588049+0000","last_became_peered":"2024-05-27T00:39:21.588049+0000","last_unstale":"2024-05-27T00:39:27.610399+0000","last_undegraded":"2024-05-27T00:39:27.610399+0000","last_fullsized":"2024-05-27T00:39:27.610399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133490+0000","last_change":"2024-05-27T00:39:21.588239+0000","last_active":"2024-05-27T00:39:28.133490+0000","last_peered":"2024-05-27T00:39:28.133490+0000","last_clean":"2024-05-27T00:39:28.133490+0000","last_became_active":"2024-05-27T00:39:21.587936+0000","last_became_peered":"2024-05-27T00:39:21.587936+0000","last_unstale":"2024-05-27T00:39:28.133490+0000","last_undegraded":"2024-05-27T00:39:28.133490+0000","last_fullsized":"2024-05-27T00:39:28.133490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604778+0000","last_change":"2024-05-27T00:39:21.591553+0000","last_active":"2024-05-27T00:39:27.604778+0000","last_peered":"2024-05-27T00:39:27.604778+0000","last_clean":"2024-05-27T00:39:27.604778+0000","last_became_active":"2024-05-27T00:39:21.591286+0000","last_became_peered":"2024-05-27T00:39:21.591286+0000","last_unstale":"2024-05-27T00:39:27.604778+0000","last_undegraded":"2024-05-27T00:39:27.604778+0000","last_fullsized":"2024-05-27T00:39:27.604778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833459+0000","last_change":"2024-05-27T00:39:21.591695+0000","last_active":"2024-05-27T00:39:28.833459+0000","last_peered":"2024-05-27T00:39:28.833459+0000","last_clean":"2024-05-27T00:39:28.833459+0000","last_became_active":"2024-05-27T00:39:21.591384+0000","last_became_peered":"2024-05-27T00:39:21.591384+0000","last_unstale":"2024-05-27T00:39:28.833459+0000","last_undegraded":"2024-05-27T00:39:28.833459+0000","last_fullsized":"2024-05-27T00:39:28.833459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:39:39.941 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-27T00:39:39.942 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T00:39:39.942 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-27T00:39:39.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-27T00:39:39.942 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-27T00:39:40.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.076+0000 7ff746189640 1 Processor -- start 2024-05-27T00:39:40.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.076+0000 7ff746189640 1 -- start start 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff746189640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7400b3630 0x7ff7400b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff746189640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7400b77e0 0x7ff740161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff746189640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74006cc70 con 0x7ff740162390 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff746189640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff7400d1e50 con 0x7ff7400b77e0 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff746189640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7400d2130 con 0x7ff7400b3630 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff737fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7400b77e0 0x7ff740161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff73ffff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff740162390 0x7ff7400d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35464/0 (socket says 172.21.15.111:35464) 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff73ffff640 1 -- 172.21.15.111:0/3106447985 learned_addr learned my addr 172.21.15.111:0/3106447985 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff73f7fe640 1 --2- 172.21.15.111:0/3106447985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7400b3630 0x7ff7400b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff737fff640 1 -- 172.21.15.111:0/3106447985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7400b3630 msgr2=0x7ff7400b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:40.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff737fff640 1 --2- 172.21.15.111:0/3106447985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7400b3630 0x7ff7400b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff737fff640 1 -- 172.21.15.111:0/3106447985 >> v1:172.21.15.111:6789/0 conn(0x7ff740162390 legacy=0x7ff7400d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff737fff640 1 -- 172.21.15.111:0/3106447985 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400d2410 con 0x7ff7400b77e0 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.077+0000 7ff73effd640 1 -- 172.21.15.111:0/3106447985 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331504880 0 0) 0x7ff74006cc70 con 0x7ff740162390 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.078+0000 7ff737fff640 1 --2- 172.21.15.111:0/3106447985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b77e0 0x7ff740161e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff7280097a0 tx=0x7ff72802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4303de1e59c8da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.078+0000 7ff73effd640 1 -- 172.21.15.111:0/3106447985 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728037070 con 0x7ff7400b77e0 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.078+0000 7ff73effd640 1 -- 172.21.15.111:0/3106447985 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff728032440 con 0x7ff7400b77e0 2024-05-27T00:39:40.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.078+0000 7ff73effd640 1 -- 172.21.15.111:0/3106447985 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728032810 con 0x7ff7400b77e0 2024-05-27T00:39:40.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.078+0000 7ff746189640 1 -- 172.21.15.111:0/3106447985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b77e0 msgr2=0x7ff740161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 --2- 172.21.15.111:0/3106447985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b77e0 0x7ff740161e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff7280097a0 tx=0x7ff72802a180 comp rx=0 tx=0).stop 2024-05-27T00:39:40.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 -- 172.21.15.111:0/3106447985 shutdown_connections 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 --2- 172.21.15.111:0/3106447985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b77e0 0x7ff740161e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 --2- 172.21.15.111:0/3106447985 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7400b3630 0x7ff7400b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 -- 172.21.15.111:0/3106447985 >> 172.21.15.111:0/3106447985 conn(0x7ff7400d42c0 msgr2=0x7ff7400d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 -- 172.21.15.111:0/3106447985 shutdown_connections 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.079+0000 7ff746189640 1 -- 172.21.15.111:0/3106447985 wait complete. 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.080+0000 7ff746189640 1 Processor -- start 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.080+0000 7ff746189640 1 -- start start 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 0x7ff74015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7400b77e0 0x7ff7400b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff740162390 0x7ff74014f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff7400d34a0 con 0x7ff7400b3630 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff7400d3320 con 0x7ff7400b77e0 2024-05-27T00:39:40.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7400d3620 con 0x7ff740162390 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 0x7ff74015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 0x7ff74015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56894/0 (socket says 172.21.15.111:56894) 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 -- 172.21.15.111:0/545368328 learned_addr learned my addr 172.21.15.111:0/545368328 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 -- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff740162390 msgr2=0x7ff74014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff740162390 0x7ff74014f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 -- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7400b77e0 msgr2=0x7ff7400b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7400b77e0 0x7ff7400b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 -- 172.21.15.111:0/545368328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff740159e80 con 0x7ff7400b3630 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73f7fe640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 0x7ff74015cf90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff73000d800 tx=0x7ff73000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000b6c0 con 0x7ff7400b3630 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff730019070 con 0x7ff7400b3630 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000ba80 con 0x7ff7400b3630 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74015a120 con 0x7ff7400b3630 2024-05-27T00:39:40.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.081+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74015a6e0 con 0x7ff7400b3630 2024-05-27T00:39:40.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.082+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff740068680 con 0x7ff7400b3630 2024-05-27T00:39:40.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.083+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff73000bc20 con 0x7ff7400b3630 2024-05-27T00:39:40.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.083+0000 7ff73cff9640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff72006be30 0x7ff72006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.083+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff73008b670 con 0x7ff7400b3630 2024-05-27T00:39:40.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.083+0000 7ff737fff640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff72006be30 0x7ff72006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.084+0000 7ff737fff640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff72006be30 0x7ff72006e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff728034000 tx=0x7ff72802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.087+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff730059ce0 con 0x7ff7400b3630 2024-05-27T00:39:40.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.304+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff7400cef10 con 0x7ff7400b3630 2024-05-27T00:39:40.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.307+0000 7ff73cff9640 1 -- 172.21.15.111:0/545368328 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172944 (secure 0 0 0) 0x7ff7300595e0 con 0x7ff7400b3630 2024-05-27T00:39:40.310 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:40.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff72006be30 msgr2=0x7ff72006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff72006be30 0x7ff72006e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff728034000 tx=0x7ff72802c000 comp rx=0 tx=0).stop 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 msgr2=0x7ff74015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 0x7ff74015cf90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff73000d800 tx=0x7ff73000dcc0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 shutdown_connections 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff72006be30 0x7ff72006e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff740162390 0x7ff74014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7400b77e0 0x7ff7400b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 --2- 172.21.15.111:0/545368328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7400b3630 0x7ff74015cf90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 >> 172.21.15.111:0/545368328 conn(0x7ff7400d42c0 msgr2=0x7ff7400b7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:40.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 shutdown_connections 2024-05-27T00:39:40.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.309+0000 7ff746189640 1 -- 172.21.15.111:0/545368328 wait complete. 2024-05-27T00:39:40.321 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":2917284088},{"type":"v1","addr":"172.21.15.153:6833","nonce":2917284088}]},"active_addr":"172.21.15.153:6833/2917284088","active_change":"2024-05-27T00:39:00.913386+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4125,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":789032760}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2902074541}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":534706773}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":1548850517}]}]} 2024-05-27T00:39:40.325 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-27T00:39:40.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-27T00:39:40.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:40.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.424+0000 7f9283295640 1 Processor -- start 2024-05-27T00:39:40.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.424+0000 7f9283295640 1 -- start start 2024-05-27T00:39:40.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9283295640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c0af970 0x7f927c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9283295640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f927c162430 0x7f927c0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9283295640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f927c0b3a20 con 0x7f927c162050 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9283295640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f927c0b4630 con 0x7f927c162430 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9283295640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f927c0b3d00 con 0x7f927c0af970 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9281832640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f927c162430 0x7f927c0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9281031640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f927c162050 0x7f927c0af260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35472/0 (socket says 172.21.15.111:35472) 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.425+0000 7f9281031640 1 -- 172.21.15.111:0/1818038669 learned_addr learned my addr 172.21.15.111:0/1818038669 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9280830640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c0af970 0x7f927c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9281832640 1 -- 172.21.15.111:0/1818038669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c0af970 msgr2=0x7f927c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9273fff640 1 -- 172.21.15.111:0/1818038669 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4191735428 0 0) 0x7f927c0b3a20 con 0x7f927c162050 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9281832640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c0af970 0x7f927c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9281832640 1 -- 172.21.15.111:0/1818038669 >> v1:172.21.15.111:6789/0 conn(0x7f927c162050 legacy=0x7f927c0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9281832640 1 -- 172.21.15.111:0/1818038669 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c0b3fe0 con 0x7f927c162430 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9280830640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c0af970 0x7f927c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:40.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9281832640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c162430 0x7f927c0b4ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92740097a0 tx=0x7f927402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78546c6f7be62a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9273fff640 1 -- 172.21.15.111:0/1818038669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274037070 con 0x7f927c162430 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9273fff640 1 -- 172.21.15.111:0/1818038669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9274032440 con 0x7f927c162430 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.426+0000 7f9273fff640 1 -- 172.21.15.111:0/1818038669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274032810 con 0x7f927c162430 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 -- 172.21.15.111:0/1818038669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c162430 msgr2=0x7f927c0b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c162430 0x7f927c0b4ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92740097a0 tx=0x7f927402a180 comp rx=0 tx=0).stop 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 -- 172.21.15.111:0/1818038669 shutdown_connections 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c162430 0x7f927c0b4ac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 --2- 172.21.15.111:0/1818038669 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c0af970 0x7f927c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 -- 172.21.15.111:0/1818038669 >> 172.21.15.111:0/1818038669 conn(0x7f927c072a30 msgr2=0x7f927c06bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 -- 172.21.15.111:0/1818038669 shutdown_connections 2024-05-27T00:39:40.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.427+0000 7f9283295640 1 -- 172.21.15.111:0/1818038669 wait complete. 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.428+0000 7f9283295640 1 Processor -- start 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.428+0000 7f9283295640 1 -- start start 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9283295640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 0x7f927c139c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9283295640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c162050 0x7f927c14e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9283295640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f927c162430 0x7f927c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9283295640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0b5180 con 0x7f927c0af970 2024-05-27T00:39:40.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9283295640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0b5000 con 0x7f927c162430 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9283295640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f927c0b5300 con 0x7f927c162050 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 0x7f927c139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 0x7f927c139c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56914/0 (socket says 172.21.15.111:56914) 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 -- 172.21.15.111:0/4002812373 learned_addr learned my addr 172.21.15.111:0/4002812373 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281832640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f927c162430 0x7f927c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9280830640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c162050 0x7f927c14e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 -- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c162050 msgr2=0x7f927c14e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c162050 0x7f927c14e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 -- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f927c162430 msgr2=0x7f927c14e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f927c162430 0x7f927c14e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281031640 1 -- 172.21.15.111:0/4002812373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1e9280 con 0x7f927c0af970 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9281832640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f927c162430 0x7f927c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:40.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.429+0000 7f9280830640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c162050 0x7f927c14e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.430+0000 7f9281031640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 0x7f927c139c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f927800c8c0 tx=0x7f927800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.430+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278018070 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.430+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927800a640 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.430+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c1e9560 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.430+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927800aa30 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.430+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1e9b20 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.431+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c068680 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.431+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f927801b070 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.432+0000 7f9271ffb640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f926006be30 0x7f926006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.432+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f927808a8b0 con 0x7f927c0af970 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.432+0000 7f9280830640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f926006be30 0x7f926006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.432+0000 7f9280830640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f926006be30 0x7f926006e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f927c0cff40 tx=0x7f926c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.436+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9278058f20 con 0x7f927c0af970 2024-05-27T00:39:40.609 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.608+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f927c0b0030 con 0x7f927c0af970 2024-05-27T00:39:40.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.609+0000 7f9271ffb640 1 -- 172.21.15.111:0/4002812373 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f9278058820 con 0x7f927c0af970 2024-05-27T00:39:40.611 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:40.611 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:40.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.611+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f926006be30 msgr2=0x7f926006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.611+0000 7f9283295640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f926006be30 0x7f926006e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f927c0cff40 tx=0x7f926c008040 comp rx=0 tx=0).stop 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.611+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 msgr2=0x7f927c139c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.611+0000 7f9283295640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 0x7f927c139c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f927800c8c0 tx=0x7f927800cd80 comp rx=0 tx=0).stop 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 shutdown_connections 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f926006be30 0x7f926006e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f927c162430 0x7f927c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f927c162050 0x7f927c14e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 --2- 172.21.15.111:0/4002812373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f927c0af970 0x7f927c139c00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 >> 172.21.15.111:0/4002812373 conn(0x7f927c072a30 msgr2=0x7f927c06df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 shutdown_connections 2024-05-27T00:39:40.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.612+0000 7f9283295640 1 -- 172.21.15.111:0/4002812373 wait complete. 2024-05-27T00:39:40.624 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-27T00:39:40.624 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:40.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.722+0000 7f729e716640 1 Processor -- start 2024-05-27T00:39:40.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.722+0000 7f729e716640 1 -- start start 2024-05-27T00:39:40.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729e716640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7298144000 0x7f72980b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729e716640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f729806b390 0x7f72980cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729e716640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72980cdc80 0x7f72980726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729e716640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f72980b42f0 con 0x7f72980cdc80 2024-05-27T00:39:40.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729e716640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f72980b3440 con 0x7f729806b390 2024-05-27T00:39:40.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729e716640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f72980b36e0 con 0x7f7298144000 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72980cdc80 0x7f72980726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f72980cdc80 0x7f72980726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50712/0 (socket says 172.21.15.111:50712) 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 -- 172.21.15.111:0/2087658278 learned_addr learned my addr 172.21.15.111:0/2087658278 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 -- 172.21.15.111:0/2087658278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7298144000 msgr2=0x7f72980b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 --2- 172.21.15.111:0/2087658278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7298144000 0x7f72980b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 -- 172.21.15.111:0/2087658278 >> v2:172.21.15.153:3300/0 conn(0x7f729806b390 msgr2=0x7f72980cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 --2- 172.21.15.111:0/2087658278 >> v2:172.21.15.153:3300/0 conn(0x7f729806b390 0x7f72980cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.723+0000 7f729ccb3640 1 -- 172.21.15.111:0/2087658278 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72980b39c0 con 0x7f72980cdc80 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.724+0000 7f729ccb3640 1 --2- 172.21.15.111:0/2087658278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72980cdc80 0x7f72980726d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f729806cd60 tx=0x7f729002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2a3ab73e435ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.724+0000 7f7296ffd640 1 -- 172.21.15.111:0/2087658278 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290036070 con 0x7f72980cdc80 2024-05-27T00:39:40.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.724+0000 7f7296ffd640 1 -- 172.21.15.111:0/2087658278 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7290031600 con 0x7f72980cdc80 2024-05-27T00:39:40.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.724+0000 7f7296ffd640 1 -- 172.21.15.111:0/2087658278 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900319d0 con 0x7f72980cdc80 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 -- 172.21.15.111:0/2087658278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72980cdc80 msgr2=0x7f72980726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 --2- 172.21.15.111:0/2087658278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72980cdc80 0x7f72980726d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f729806cd60 tx=0x7f729002a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 -- 172.21.15.111:0/2087658278 shutdown_connections 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 --2- 172.21.15.111:0/2087658278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72980cdc80 0x7f72980726d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 --2- 172.21.15.111:0/2087658278 >> v2:172.21.15.153:3300/0 conn(0x7f729806b390 0x7f72980cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 --2- 172.21.15.111:0/2087658278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7298144000 0x7f72980b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 -- 172.21.15.111:0/2087658278 >> 172.21.15.111:0/2087658278 conn(0x7f72980b1c00 msgr2=0x7f72980b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 -- 172.21.15.111:0/2087658278 shutdown_connections 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.725+0000 7f729e716640 1 -- 172.21.15.111:0/2087658278 wait complete. 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.726+0000 7f729e716640 1 Processor -- start 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.726+0000 7f729e716640 1 -- start start 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729e716640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f729806b390 0x7f7298155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729e716640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72980cdc80 0x7f72981f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729e716640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 0x7f72980ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729e716640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7298072c10 con 0x7f7298144000 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729e716640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f72980b3f80 con 0x7f72980cdc80 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729e716640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7298072d90 con 0x7f729806b390 2024-05-27T00:39:40.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 0x7f72980ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 0x7f72980ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50714/0 (socket says 172.21.15.111:50714) 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 -- 172.21.15.111:0/2679385962 learned_addr learned my addr 172.21.15.111:0/2679385962 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f72977fe640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72980cdc80 0x7f72981f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f7297fff640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f729806b390 0x7f7298155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 -- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f729806b390 msgr2=0x7f7298155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f729806b390 0x7f7298155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 -- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72980cdc80 msgr2=0x7f72981f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72980cdc80 0x7f72981f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 -- 172.21.15.111:0/2679385962 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72981362b0 con 0x7f7298144000 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f72977fe640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72980cdc80 0x7f72981f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:40.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f7297fff640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f729806b390 0x7f7298155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.727+0000 7f729ccb3640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 0x7f72980ce9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7290009f40 tx=0x7f7290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.728+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290041020 con 0x7f7298144000 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.728+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7290007780 con 0x7f7298144000 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.728+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290036040 con 0x7f7298144000 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.728+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7298136550 con 0x7f7298144000 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.728+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7298136b10 con 0x7f7298144000 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.729+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298068680 con 0x7f7298144000 2024-05-27T00:39:40.733 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.729+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7290044020 con 0x7f7298144000 2024-05-27T00:39:40.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.730+0000 7f7294ff9640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f728406be30 0x7f728406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:40.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.730+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f72900abf70 con 0x7f7298144000 2024-05-27T00:39:40.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.730+0000 7f7297fff640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f728406be30 0x7f728406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:40.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.730+0000 7f7297fff640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f728406be30 0x7f728406e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72981360d0 tx=0x7f7288008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:40.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.734+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f729004b030 con 0x7f7298144000 2024-05-27T00:39:40.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.908+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7298071eb0 con 0x7f7298144000 2024-05-27T00:39:40.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.909+0000 7f7294ff9640 1 -- 172.21.15.111:0/2679385962 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f729007a730 con 0x7f7298144000 2024-05-27T00:39:40.910 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:40.910 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f728406be30 msgr2=0x7f728406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f728406be30 0x7f728406e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72981360d0 tx=0x7f7288008040 comp rx=0 tx=0).stop 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 msgr2=0x7f72980ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 0x7f72980ce9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7290009f40 tx=0x7f7290004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 shutdown_connections 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f728406be30 0x7f728406e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7298144000 0x7f72980ce9a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72980cdc80 0x7f72981f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 --2- 172.21.15.111:0/2679385962 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f729806b390 0x7f7298155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:40.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 >> 172.21.15.111:0/2679385962 conn(0x7f72980b1c00 msgr2=0x7f72980af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:40.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 shutdown_connections 2024-05-27T00:39:40.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:40.911+0000 7f729e716640 1 -- 172.21.15.111:0/2679385962 wait complete. 2024-05-27T00:39:40.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-27T00:39:40.924 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-27T00:39:40.924 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-27T00:39:40.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-27T00:39:40.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-27T00:39:40.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-27T00:39:40.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-27T00:39:40.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-27T00:39:41.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.096+0000 7f3795458640 1 Processor -- start 2024-05-27T00:39:41.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.096+0000 7f9a07aba640 1 Processor -- start 2024-05-27T00:39:41.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.096+0000 7f3795458640 1 -- start start 2024-05-27T00:39:41.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f774fcb3640 1 Processor -- start 2024-05-27T00:39:41.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.096+0000 7f9a07aba640 1 -- start start 2024-05-27T00:39:41.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f774fcb3640 1 -- start start 2024-05-27T00:39:41.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f2f0b59e640 1 Processor -- start 2024-05-27T00:39:41.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a07aba640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9a0006bda0 0x7f9a000d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a07aba640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a000d1f60 con 0x7f9a000d3520 2024-05-27T00:39:41.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a07aba640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a000d2220 con 0x7f9a001627d0 2024-05-27T00:39:41.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a07aba640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9a00162bb0 con 0x7f9a0006bda0 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f3795458640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f379014b370 0x7f3790072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f3795458640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 0x7f37900b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f3795458640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37900b3750 0x7f37900b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f3795458640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f37900cd4f0 con 0x7f37900b3750 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f3795458640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f379014b750 con 0x7f379014b370 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f3795458640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f37900b3be0 con 0x7f379014c020 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f3787fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 0x7f37900b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f3794c57640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f37900b3750 0x7f37900b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f3787fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 0x7f37900b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47624/0 (socket says 172.21.15.111:47624) 2024-05-27T00:39:41.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f3787fff640 1 -- 172.21.15.111:0/3920384160 learned_addr learned my addr 172.21.15.111:0/3920384160 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a06057640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9a000d3520 0x7f9a001662c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36634/0 (socket says 172.21.15.111:36634) 2024-05-27T00:39:41.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a06057640 1 -- 172.21.15.111:0/271936653 learned_addr learned my addr 172.21.15.111:0/271936653 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.097+0000 7f9a365c8640 1 Processor -- start 2024-05-27T00:39:41.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a365c8640 1 -- start start 2024-05-27T00:39:41.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f378ffff640 1 --2- 172.21.15.111:0/3920384160 >> v2:172.21.15.153:3300/0 conn(0x7f379014b370 0x7f3790072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f774fcb3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740034050 0x7f774004fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7fe3627d5640 1 Processor -- start 2024-05-27T00:39:41.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f06118a7640 1 Processor -- start 2024-05-27T00:39:41.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f9a05055640 1 --2- 172.21.15.111:0/271936653 >> v2:172.21.15.153:3300/0 conn(0x7f9a0006bda0 0x7f9a000d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f774fcb3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7740054c40 con 0x7f774004b3f0 2024-05-27T00:39:41.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4255685723 0 0) 0x7f9a000d1f60 con 0x7f9a000d3520 2024-05-27T00:39:41.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f774fcb3640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7740053a80 con 0x7f7740006a80 2024-05-27T00:39:41.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99ec003520 con 0x7f9a000d3520 2024-05-27T00:39:41.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f774fcb3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7740006310 con 0x7f7740034050 2024-05-27T00:39:41.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f774da4f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f774004b3f0 0x7f774004f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36636/0 (socket says 172.21.15.111:36636) 2024-05-27T00:39:41.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f774da4f640 1 -- 172.21.15.111:0/3277696724 learned_addr learned my addr 172.21.15.111:0/3277696724 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82241871 0 0) 0x7f9a000d2220 con 0x7f9a001627d0 2024-05-27T00:39:41.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f06118a7640 1 -- start start 2024-05-27T00:39:41.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f774d24e640 1 --2- 172.21.15.111:0/3277696724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740034050 0x7f774004fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a000d1f60 con 0x7f9a001627d0 2024-05-27T00:39:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f774ca4d640 1 -- 172.21.15.111:0/3277696724 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531620744 0 0) 0x7f7740053a80 con 0x7f7740006a80 2024-05-27T00:39:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f3787fff640 1 -- 172.21.15.111:0/3920384160 >> v2:172.21.15.153:3300/0 conn(0x7f379014b370 msgr2=0x7f3790072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f3787fff640 1 --2- 172.21.15.111:0/3920384160 >> v2:172.21.15.153:3300/0 conn(0x7f379014b370 0x7f3790072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f3787fff640 1 -- 172.21.15.111:0/3920384160 >> v2:172.21.15.111:3300/0 conn(0x7f37900b3750 msgr2=0x7f37900b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f3787fff640 1 --2- 172.21.15.111:0/3920384160 >> v2:172.21.15.111:3300/0 conn(0x7f37900b3750 0x7f37900b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f3787fff640 1 -- 172.21.15.111:0/3920384160 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37900b3ec0 con 0x7f379014c020 2024-05-27T00:39:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f3794c57640 1 --2- 172.21.15.111:0/3920384160 >> v2:172.21.15.111:3300/0 conn(0x7f37900b3750 0x7f37900b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2991127237 0 0) 0x7f99ec003520 con 0x7f9a000d3520 2024-05-27T00:39:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 >> v2:172.21.15.153:3300/0 conn(0x7f9a0006bda0 msgr2=0x7f9a000d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f774ca4d640 1 -- 172.21.15.111:0/3277696724 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7734003520 con 0x7f7740006a80 2024-05-27T00:39:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 --2- 172.21.15.111:0/271936653 >> v2:172.21.15.153:3300/0 conn(0x7f9a0006bda0 0x7f9a000d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 >> v1:172.21.15.153:6789/0 conn(0x7f9a001627d0 legacy=0x7f9a000b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a000d2500 con 0x7f9a000d3520 2024-05-27T00:39:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.098+0000 7f2f0b59e640 1 -- start start 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0b59e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040069e0 0x7f2f04006e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0b59e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f04053720 con 0x7f2f040ead00 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0b59e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f04053a00 con 0x7f2f04005e90 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0b59e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2f04054330 con 0x7f2f040069e0 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f774d24e640 1 -- 172.21.15.111:0/3277696724 >> v1:172.21.15.153:6789/0 conn(0x7f7740006a80 legacy=0x7f7740006f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f774d24e640 1 -- 172.21.15.111:0/3277696724 >> v1:172.21.15.111:6789/0 conn(0x7f774004b3f0 legacy=0x7f774004f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f774d24e640 1 -- 172.21.15.111:0/3277696724 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7740053d60 con 0x7f7740034050 2024-05-27T00:39:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0a59c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2f040ead00 0x7f2f0404fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36640/0 (socket says 172.21.15.111:36640) 2024-05-27T00:39:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0a59c640 1 -- 172.21.15.111:0/2690253258 learned_addr learned my addr 172.21.15.111:0/2690253258 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f99f8001970 con 0x7f9a000d3520 2024-05-27T00:39:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 >> v1:172.21.15.111:6789/0 conn(0x7f9a000d3520 legacy=0x7f9a001662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f06118a7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 0x7f060404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0ad9d640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040069e0 0x7f2f04006e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545603549 0 0) 0x7f2f04053720 con 0x7f2f040ead00 2024-05-27T00:39:41.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a04854640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99ec003b90 0x7f99ec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ef4003520 con 0x7f2f040ead00 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f62aba5d640 1 Processor -- start 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7f62aba5d640 1 -- start start 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f62aba5d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f62a41637e0 0x7f62a40b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f62aba5d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62a4071f00 0x7f62a414b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f62aba5d640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a40b4420 con 0x7f62a40b3f10 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f62aba5d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f62a40b4700 con 0x7f62a41637e0 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f62aba5d640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f62a40b49e0 con 0x7f62a4071f00 2024-05-27T00:39:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f06118a7640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0604006ea0 0x7f0604007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a04854640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f99ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.099+0000 7fe3627d5640 1 -- start start 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f3787fff640 1 --2- 172.21.15.111:0/3920384160 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 0x7f37900b4be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3790073700 tx=0x7f377802a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542c6bbb8ff973a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a365c8640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a3006c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a355c6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a3006c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a355c6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a3006c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47648/0 (socket says 172.21.15.111:47648) 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a365c8640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9a300b5600 0x7f9a30163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a355c6640 1 -- 172.21.15.111:0/61192159 learned_addr learned my addr 172.21.15.111:0/61192159 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a30163a60 con 0x7f9a300ce670 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9a300cd8b0 con 0x7f9a300b5600 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9a300cdb90 con 0x7f9a300b5220 2024-05-27T00:39:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.100+0000 7f9a35dc7640 1 --2- 172.21.15.111:0/61192159 >> v2:172.21.15.153:3300/0 conn(0x7f9a300b5600 0x7f9a30163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a05856640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99ec003b90 0x7f99ec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a9ffa640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62a4071f00 0x7f62a414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a9ffa640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f62a4071f00 0x7f62a414b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56214/0 (socket says 172.21.15.111:56214) 2024-05-27T00:39:41.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a9ffa640 1 -- 172.21.15.111:0/4282462244 learned_addr learned my addr 172.21.15.111:0/4282462244 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe3627d5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c0b5e40 0x7fe35c15eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe3627d5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe35c15f680 con 0x7fe35c15bed0 2024-05-27T00:39:41.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe3627d5640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe35c06d4e0 con 0x7fe35c06dfc0 2024-05-27T00:39:41.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe3627d5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c15f960 con 0x7fe35c0b5e40 2024-05-27T00:39:41.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a97f9640 1 --2- 172.21.15.111:0/4282462244 >> v2:172.21.15.111:3300/0 conn(0x7f62a41637e0 0x7f62a40b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f378f7fe640 1 -- 172.21.15.111:0/3920384160 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778037070 con 0x7f379014c020 2024-05-27T00:39:41.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe35bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe35c15bed0 0x7fe35c0b5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36644/0 (socket says 172.21.15.111:36644) 2024-05-27T00:39:41.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe35bfff640 1 -- 172.21.15.111:0/329389822 learned_addr learned my addr 172.21.15.111:0/329389822 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3945050337 0 0) 0x7f2ef4003520 con 0x7f2f040ead00 2024-05-27T00:39:41.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040069e0 msgr2=0x7f2f04006e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a35dc7640 1 -- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 msgr2=0x7f9a3006c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a35dc7640 1 --2- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a3006c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a35dc7640 1 -- 172.21.15.111:0/61192159 >> v1:172.21.15.153:6789/0 conn(0x7f9a300ce670 legacy=0x7f9a30162cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a35dc7640 1 -- 172.21.15.111:0/61192159 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a300cde70 con 0x7f9a300b5600 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040069e0 0x7f2f04006e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 >> v1:172.21.15.153:6789/0 conn(0x7f2f04005e90 legacy=0x7f2f040062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f04053ce0 con 0x7f2f040ead00 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f2f0c04ffb0 con 0x7f2f040ead00 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 >> v1:172.21.15.111:6789/0 conn(0x7f2f040ead00 legacy=0x7f2f0404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ef40039e0 0x7f2ef4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f378f7fe640 1 -- 172.21.15.111:0/3920384160 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3778032440 con 0x7f379014c020 2024-05-27T00:39:41.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4004800 0x7f2ef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef40051c0 0x7f2ef400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04054330 con 0x7f2ef40051c0 2024-05-27T00:39:41.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef400bf00 con 0x7f2ef4004800 2024-05-27T00:39:41.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a04854640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec005210 0x7f99ec00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef400c0b0 con 0x7f2ef40039e0 2024-05-27T00:39:41.120 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7fe35b7fe640 1 --2- 172.21.15.111:0/329389822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c0b5e40 0x7fe35c15eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a97f9640 1 -- 172.21.15.111:0/4282462244 >> v2:172.21.15.153:3300/0 conn(0x7f62a4071f00 msgr2=0x7f62a414b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a97f9640 1 --2- 172.21.15.111:0/4282462244 >> v2:172.21.15.153:3300/0 conn(0x7f62a4071f00 0x7f62a414b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a97f9640 1 -- 172.21.15.111:0/4282462244 >> v1:172.21.15.111:6789/0 conn(0x7f62a40b3f10 legacy=0x7f62a40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec00c190 con 0x7f99ec004800 2024-05-27T00:39:41.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a05055640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f99ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f774d24e640 1 --2- 172.21.15.111:0/3277696724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740034050 0x7f774004fc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77440097d0 tx=0x7f774402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc25dbf462831412 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f62a97f9640 1 -- 172.21.15.111:0/4282462244 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a40b4cc0 con 0x7f62a41637e0 2024-05-27T00:39:41.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ef40039e0 0x7f2ef4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f06108a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 0x7f060404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f06108a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 0x7f060404fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47662/0 (socket says 172.21.15.111:47662) 2024-05-27T00:39:41.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f060bfff640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f060404ffd0 0x7f0604037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47240/0 (socket says 172.21.15.111:47240) 2024-05-27T00:39:41.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f060bfff640 1 -- 172.21.15.111:0/3412454549 learned_addr learned my addr 172.21.15.111:0/3412454549 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0604053720 con 0x7f060404ffd0 2024-05-27T00:39:41.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0604054330 con 0x7f0604006ea0 2024-05-27T00:39:41.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0604053a00 con 0x7f06040ead00 2024-05-27T00:39:41.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f378f7fe640 1 -- 172.21.15.111:0/3920384160 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780327f0 con 0x7f379014c020 2024-05-27T00:39:41.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f774ca4d640 1 -- 172.21.15.111:0/3277696724 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744037070 con 0x7f7740034050 2024-05-27T00:39:41.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7fe35b7fe640 1 -- 172.21.15.111:0/329389822 >> v1:172.21.15.153:6789/0 conn(0x7fe35c06dfc0 legacy=0x7fe35c06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7fe35b7fe640 1 -- 172.21.15.111:0/329389822 >> v1:172.21.15.111:6789/0 conn(0x7fe35c15bed0 legacy=0x7fe35c0b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f09d9b640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4004800 0x7f2ef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f774ca4d640 1 -- 172.21.15.111:0/3277696724 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7744032440 con 0x7f7740034050 2024-05-27T00:39:41.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f774ca4d640 1 -- 172.21.15.111:0/3277696724 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744032810 con 0x7f7740034050 2024-05-27T00:39:41.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7fe35b7fe640 1 -- 172.21.15.111:0/329389822 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c06d7c0 con 0x7fe35c0b5e40 2024-05-27T00:39:41.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f060b7fe640 1 -- 172.21.15.111:0/3412454549 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734875575 0 0) 0x7f0604053720 con 0x7f060404ffd0 2024-05-27T00:39:41.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f060b7fe640 1 -- 172.21.15.111:0/3412454549 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05f4003520 con 0x7f060404ffd0 2024-05-27T00:39:41.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06110a6640 1 --2- 172.21.15.111:0/3412454549 >> v2:172.21.15.153:3300/0 conn(0x7f0604006ea0 0x7f0604007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f0ad9d640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef40051c0 0x7f2ef400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 -- 172.21.15.111:0/3920384160 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 msgr2=0x7f37900b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06108a5640 1 -- 172.21.15.111:0/3412454549 >> v2:172.21.15.153:3300/0 conn(0x7f0604006ea0 msgr2=0x7f0604007330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06108a5640 1 --2- 172.21.15.111:0/3412454549 >> v2:172.21.15.153:3300/0 conn(0x7f0604006ea0 0x7f0604007330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06108a5640 1 -- 172.21.15.111:0/3412454549 >> v1:172.21.15.153:6789/0 conn(0x7f060404ffd0 legacy=0x7f0604037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06108a5640 1 -- 172.21.15.111:0/3412454549 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0604053ce0 con 0x7f06040ead00 2024-05-27T00:39:41.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 --2- 172.21.15.111:0/3920384160 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 0x7f37900b4be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3790073700 tx=0x7f377802a160 comp rx=0 tx=0).stop 2024-05-27T00:39:41.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f06110a6640 1 --2- 172.21.15.111:0/3412454549 >> v2:172.21.15.153:3300/0 conn(0x7f0604006ea0 0x7f0604007330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:41.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 -- 172.21.15.111:0/3920384160 shutdown_connections 2024-05-27T00:39:41.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 --2- 172.21.15.111:0/3920384160 >> v2:172.21.15.111:3300/0 conn(0x7f37900b3750 0x7f37900b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec00c340 con 0x7f99ec005210 2024-05-27T00:39:41.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f99ec00c540 con 0x7f99ec003b90 2024-05-27T00:39:41.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f9a05055640 1 -- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99ec003b90 msgr2=0x7f99ec003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f9a05055640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99ec003b90 0x7f99ec003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f9a05055640 1 -- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec005210 msgr2=0x7f99ec00ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 --2- 172.21.15.111:0/3920384160 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f379014c020 0x7f37900b4be0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 --2- 172.21.15.111:0/3920384160 >> v2:172.21.15.153:3300/0 conn(0x7f379014b370 0x7f3790072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f3795458640 1 -- 172.21.15.111:0/3920384160 >> 172.21.15.111:0/3920384160 conn(0x7f379006d300 msgr2=0x7f379006d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f62a97f9640 1 --2- 172.21.15.111:0/4282462244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a41637e0 0x7f62a40b39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62a4159d60 tx=0x7f629402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8d1f6523dcc333 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 -- 172.21.15.111:0/3277696724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740034050 msgr2=0x7f774004fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 --2- 172.21.15.111:0/3277696724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740034050 0x7f774004fc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f77440097d0 tx=0x7f774402a1c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 -- 172.21.15.111:0/3277696724 shutdown_connections 2024-05-27T00:39:41.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 --2- 172.21.15.111:0/3277696724 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740034050 0x7f774004fc10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 -- 172.21.15.111:0/3277696724 >> 172.21.15.111:0/3277696724 conn(0x7f774000a430 msgr2=0x7f774000a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f0a59c640 1 -- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4004800 msgr2=0x7f2ef4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef4004800 0x7f2ef4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f0a59c640 1 -- 172.21.15.111:0/2690253258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef40051c0 msgr2=0x7f2ef400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef40051c0 0x7f2ef400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.102+0000 7f2f0a59c640 1 -- 172.21.15.111:0/2690253258 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f04053ce0 con 0x7f2ef40039e0 2024-05-27T00:39:41.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.101+0000 7f9a06057640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec005210 0x7f99ec00ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 -- 172.21.15.111:0/3277696724 shutdown_connections 2024-05-27T00:39:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f3795458640 1 -- 172.21.15.111:0/3920384160 shutdown_connections 2024-05-27T00:39:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a35dc7640 1 --2- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a30163400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a30162b30 tx=0x7f9a2002a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f53520411411aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f629bfff640 1 -- 172.21.15.111:0/4282462244 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294037070 con 0x7f62a41637e0 2024-05-27T00:39:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f629bfff640 1 -- 172.21.15.111:0/4282462244 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6294032440 con 0x7f62a41637e0 2024-05-27T00:39:41.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a05055640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec005210 0x7f99ec00ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a27fff640 1 -- 172.21.15.111:0/61192159 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20037070 con 0x7f9a300b5600 2024-05-27T00:39:41.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a27fff640 1 -- 172.21.15.111:0/61192159 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2002adb0 con 0x7f9a300b5600 2024-05-27T00:39:41.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a27fff640 1 -- 172.21.15.111:0/61192159 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20032660 con 0x7f9a300b5600 2024-05-27T00:39:41.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a05055640 1 -- 172.21.15.111:0/271936653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec003520 con 0x7f99ec004800 2024-05-27T00:39:41.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a05856640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99ec003b90 0x7f99ec003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:41.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ef40039e0 0x7f2ef4003dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c050510 tx=0x7f2ef4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f4e6cfdee792c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a06057640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ec005210 0x7f99ec00ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:41.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f3795458640 1 -- 172.21.15.111:0/3920384160 wait complete. 2024-05-27T00:39:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7fe35b7fe640 1 --2- 172.21.15.111:0/329389822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c0b5e40 0x7fe35c15eaa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0097d0 tx=0x7fe34c029fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e19dd7a8fd9e48 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f774fcb3640 1 -- 172.21.15.111:0/3277696724 wait complete. 2024-05-27T00:39:41.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f9a06057640 1 -- 172.21.15.111:0/271936653 reap_dead start 2024-05-27T00:39:41.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c068b00 con 0x7f2ef40039e0 2024-05-27T00:39:41.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c0aa070 con 0x7f2ef40039e0 2024-05-27T00:39:41.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.103+0000 7f629bfff640 1 -- 172.21.15.111:0/4282462244 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294032810 con 0x7f62a41637e0 2024-05-27T00:39:41.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f2f0959a640 1 -- 172.21.15.111:0/2690253258 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0a6c50 con 0x7f2ef40039e0 2024-05-27T00:39:41.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f62aba5d640 1 -- 172.21.15.111:0/4282462244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a41637e0 msgr2=0x7f62a40b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7fe35affd640 1 -- 172.21.15.111:0/329389822 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c037070 con 0x7fe35c0b5e40 2024-05-27T00:39:41.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f9a05055640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f99ec004c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99f000ba30 tx=0x7f99ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f2ada96704fc32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f62aba5d640 1 --2- 172.21.15.111:0/4282462244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a41637e0 0x7f62a40b39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62a4159d60 tx=0x7f629402a1a0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7fe35affd640 1 -- 172.21.15.111:0/329389822 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe34c02ac70 con 0x7fe35c0b5e40 2024-05-27T00:39:41.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f06108a5640 1 --2- 172.21.15.111:0/3412454549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 0x7f060404fa90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06000097a0 tx=0x7f060002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29355b9c3785e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f2f0b59e640 1 -- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ef40039e0 msgr2=0x7f2ef4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f2f0b59e640 1 --2- 172.21.15.111:0/2690253258 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2ef40039e0 0x7f2ef4003dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c050510 tx=0x7f2ef4005830 comp rx=0 tx=0).stop 2024-05-27T00:39:41.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.104+0000 7f2f0ad9d640 1 -- 172.21.15.111:0/2690253258 reap_dead start 2024-05-27T00:39:41.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7fe35affd640 1 -- 172.21.15.111:0/329389822 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c032660 con 0x7fe35c0b5e40 2024-05-27T00:39:41.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f060b7fe640 1 -- 172.21.15.111:0/3412454549 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600037070 con 0x7f06040ead00 2024-05-27T00:39:41.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0030070 con 0x7f99ec004800 2024-05-27T00:39:41.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f62aba5d640 1 -- 172.21.15.111:0/4282462244 shutdown_connections 2024-05-27T00:39:41.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f62aba5d640 1 --2- 172.21.15.111:0/4282462244 >> v2:172.21.15.153:3300/0 conn(0x7f62a4071f00 0x7f62a414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f060b7fe640 1 -- 172.21.15.111:0/3412454549 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0600032440 con 0x7f06040ead00 2024-05-27T00:39:41.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f62aba5d640 1 --2- 172.21.15.111:0/4282462244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a41637e0 0x7f62a40b39d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f62aba5d640 1 -- 172.21.15.111:0/4282462244 >> 172.21.15.111:0/4282462244 conn(0x7f62a406b390 msgr2=0x7f62a406b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f060b7fe640 1 -- 172.21.15.111:0/3412454549 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600032810 con 0x7f06040ead00 2024-05-27T00:39:41.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 msgr2=0x7f9a30163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 --2- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a30163400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a30162b30 tx=0x7f9a2002a0a0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f3795458640 1 Processor -- start 2024-05-27T00:39:41.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f62aba5d640 1 -- 172.21.15.111:0/4282462244 shutdown_connections 2024-05-27T00:39:41.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a04854640 1 -- 172.21.15.111:0/271936653 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f002add0 con 0x7f99ec004800 2024-05-27T00:39:41.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 shutdown_connections 2024-05-27T00:39:41.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 --2- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a30163400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 --2- 172.21.15.111:0/61192159 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a3006c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 >> 172.21.15.111:0/61192159 conn(0x7f9a300723e0 msgr2=0x7f9a300727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f3795458640 1 -- start start 2024-05-27T00:39:41.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f2f0b59e640 1 -- 172.21.15.111:0/2690253258 shutdown_connections 2024-05-27T00:39:41.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f2f0b59e640 1 -- 172.21.15.111:0/2690253258 >> 172.21.15.111:0/2690253258 conn(0x7f2f04009fb0 msgr2=0x7f2f0400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f62aba5d640 1 -- 172.21.15.111:0/4282462244 wait complete. 2024-05-27T00:39:41.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f774fcb3640 1 Processor -- start 2024-05-27T00:39:41.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f774fcb3640 1 -- start start 2024-05-27T00:39:41.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 -- 172.21.15.111:0/329389822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c0b5e40 msgr2=0x7fe35c15eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 --2- 172.21.15.111:0/329389822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c0b5e40 0x7fe35c15eaa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0097d0 tx=0x7fe34c029fe0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 -- 172.21.15.111:0/329389822 shutdown_connections 2024-05-27T00:39:41.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 --2- 172.21.15.111:0/329389822 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c0b5e40 0x7fe35c15eaa0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0097d0 tx=0x7fe34c029fe0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 msgr2=0x7f060404fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 -- 172.21.15.111:0/329389822 >> 172.21.15.111:0/329389822 conn(0x7fe35c073130 msgr2=0x7fe35c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f06118a7640 1 --2- 172.21.15.111:0/3412454549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 0x7f060404fa90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06000097a0 tx=0x7f060002a180 comp rx=0 tx=0).stop 2024-05-27T00:39:41.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 shutdown_connections 2024-05-27T00:39:41.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f9a07aba640 1 -- 172.21.15.111:0/271936653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 msgr2=0x7f99ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f9a07aba640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f99ec004c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99f000ba30 tx=0x7f99ec005830 comp rx=0 tx=0).stop 2024-05-27T00:39:41.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f06118a7640 1 --2- 172.21.15.111:0/3412454549 >> v2:172.21.15.153:3300/0 conn(0x7f0604006ea0 0x7f0604007330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3795458640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379014b370 0x7f37901f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3795458640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379014c020 0x7f3790149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f2f0b59e640 1 -- 172.21.15.111:0/2690253258 shutdown_connections 2024-05-27T00:39:41.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3795458640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37901498b0 0x7f3790159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3795458640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3790071410 con 0x7f379014c020 2024-05-27T00:39:41.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3795458640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3790071130 con 0x7f379014b370 2024-05-27T00:39:41.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3795458640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f37901661b0 con 0x7f37901498b0 2024-05-27T00:39:41.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f06118a7640 1 --2- 172.21.15.111:0/3412454549 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f06040ead00 0x7f060404fa90 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06000097a0 tx=0x7f060002a180 comp rx=0 tx=0).stop 2024-05-27T00:39:41.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 >> 172.21.15.111:0/3412454549 conn(0x7f0604009fb0 msgr2=0x7f060400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3787fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379014c020 0x7f3790149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3787fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379014c020 0x7f3790149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50758/0 (socket says 172.21.15.111:50758) 2024-05-27T00:39:41.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3787fff640 1 -- 172.21.15.111:0/71825070 learned_addr learned my addr 172.21.15.111:0/71825070 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 -- 172.21.15.111:0/329389822 shutdown_connections 2024-05-27T00:39:41.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f378ffff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379014b370 0x7f37901f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f774fcb3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740006a80 0x7f774004e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f3794c57640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37901498b0 0x7f3790159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f774fcb3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7740034050 0x7f77400df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774fcb3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f774004b3f0 0x7f77400e4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3787fff640 1 -- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37901498b0 msgr2=0x7f3790159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3787fff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37901498b0 0x7f3790159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3787fff640 1 -- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379014b370 msgr2=0x7f37901f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3787fff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379014b370 0x7f37901f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3787fff640 1 -- 172.21.15.111:0/71825070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379014a060 con 0x7f379014c020 2024-05-27T00:39:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f378ffff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f379014b370 0x7f37901f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:41.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774da4f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740006a80 0x7f774004e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774da4f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740006a80 0x7f774004e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47688/0 (socket says 172.21.15.111:47688) 2024-05-27T00:39:41.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7fe3627d5640 1 -- 172.21.15.111:0/329389822 wait complete. 2024-05-27T00:39:41.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774d24e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7740034050 0x7f77400df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774d24e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7740034050 0x7f77400df6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56246/0 (socket says 172.21.15.111:56246) 2024-05-27T00:39:41.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.106+0000 7f2f0b59e640 1 -- 172.21.15.111:0/2690253258 wait complete. 2024-05-27T00:39:41.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774fcb3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f774004b7d0 con 0x7f774004b3f0 2024-05-27T00:39:41.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774fcb3640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7740050150 con 0x7f7740034050 2024-05-27T00:39:41.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774fcb3640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7740054860 con 0x7f7740006a80 2024-05-27T00:39:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774da4f640 1 -- 172.21.15.111:0/371951923 learned_addr learned my addr 172.21.15.111:0/371951923 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3787fff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379014c020 0x7f3790149370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f377802a1a0 tx=0x7f37780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f62aba5d640 1 Processor -- start 2024-05-27T00:39:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f62aba5d640 1 -- start start 2024-05-27T00:39:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778037040 con 0x7f379014c020 2024-05-27T00:39:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774e250640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f774004b3f0 0x7f77400e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774da4f640 1 -- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7740034050 msgr2=0x7f77400df6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3795458640 1 -- 172.21.15.111:0/71825070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379014a340 con 0x7f379014c020 2024-05-27T00:39:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f3795458640 1 -- 172.21.15.111:0/71825070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3790159010 con 0x7f379014c020 2024-05-27T00:39:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774da4f640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7740034050 0x7f77400df6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f774da4f640 1 -- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f774004b3f0 msgr2=0x7f77400e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f774da4f640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f774004b3f0 0x7f77400e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f774da4f640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774004ce20 con 0x7f7740006a80 2024-05-27T00:39:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3778003c40 con 0x7f379014c020 2024-05-27T00:39:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 shutdown_connections 2024-05-27T00:39:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f774e250640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f774004b3f0 0x7f77400e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a07aba640 1 -- 172.21.15.111:0/271936653 shutdown_connections 2024-05-27T00:39:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a07aba640 1 --2- 172.21.15.111:0/271936653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f99ec004c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a07aba640 1 -- 172.21.15.111:0/271936653 >> 172.21.15.111:0/271936653 conn(0x7f9a000d4570 msgr2=0x7f9a000d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f2f0b59e640 1 Processor -- start 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f2f0b59e640 1 -- start start 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f2f0b59e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef40039e0 0x7f2f0404e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f774da4f640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740006a80 0x7f774004e1d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c3f0 tx=0x7f773c00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c017070 con 0x7f7740006a80 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773c005ce0 con 0x7f7740006a80 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c0072f0 con 0x7f7740006a80 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f774fcb3640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7740037d80 con 0x7f7740006a80 2024-05-27T00:39:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f774fcb3640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7740038340 con 0x7f7740006a80 2024-05-27T00:39:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780325f0 con 0x7f379014c020 2024-05-27T00:39:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a07aba640 1 -- 172.21.15.111:0/271936653 shutdown_connections 2024-05-27T00:39:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7fe3627d5640 1 Processor -- start 2024-05-27T00:39:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7fe3627d5640 1 -- start start 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f06118a7640 1 -- 172.21.15.111:0/3412454549 wait complete. 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 shutdown_connections 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.105+0000 7f9a365c8640 1 -- 172.21.15.111:0/61192159 wait complete. 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f9a365c8640 1 Processor -- start 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.107+0000 7f9a365c8640 1 -- start start 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a365c8640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a300af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a365c8640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a300afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a365c8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a300ce670 0x7f9a301f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a355c6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a300af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a355c6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a300af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47692/0 (socket says 172.21.15.111:47692) 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f9a365c8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3006cdb0 con 0x7f9a300ce670 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f9a34dc5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a300afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f9a34dc5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a300afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56254/0 (socket says 172.21.15.111:56254) 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.109+0000 7f9a34dc5640 1 -- 172.21.15.111:0/2660685201 learned_addr learned my addr 172.21.15.111:0/2660685201 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0b59e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040e9350 0x7f2f040365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe3627d5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c06dfc0 0x7fe35c141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe3627d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe35c15bed0 0x7fe35c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef40039e0 0x7f2f0404e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0a59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef40039e0 0x7f2f0404e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56270/0 (socket says 172.21.15.111:56270) 2024-05-27T00:39:41.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0a59c640 1 -- 172.21.15.111:0/1615949821 learned_addr learned my addr 172.21.15.111:0/1615949821 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f3795458640 1 -- 172.21.15.111:0/71825070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3774001030 con 0x7f379014c020 2024-05-27T00:39:41.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0b59e640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f040e97d0 0x7f2f04036b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe3627d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe35c154d00 0x7fe35c142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f09d9b640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040e9350 0x7f2f040365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.108+0000 7f9a07aba640 1 -- 172.21.15.111:0/271936653 wait complete. 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe35b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe35c15bed0 0x7fe35c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe3627d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c15c430 con 0x7fe35c15bed0 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe35b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe35c15bed0 0x7fe35c0cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50788/0 (socket says 172.21.15.111:50788) 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe3627d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c15c2b0 con 0x7fe35c154d00 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe35b7fe640 1 -- 172.21.15.111:0/2295367975 learned_addr learned my addr 172.21.15.111:0/2295367975 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c15efe0 con 0x7fe35c06dfc0 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0b59e640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040547c0 con 0x7f2f040e97d0 2024-05-27T00:39:41.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0b59e640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040501a0 con 0x7f2ef40039e0 2024-05-27T00:39:41.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3778032790 con 0x7f379014c020 2024-05-27T00:39:41.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0b59e640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2f04054940 con 0x7f2f040e9350 2024-05-27T00:39:41.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f9a07aba640 1 Processor -- start 2024-05-27T00:39:41.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62aba5d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4071f00 0x7f62a41378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62aba5d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62a40b3f10 0x7f62a41397f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62aba5d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62a41637e0 0x7f62a4165e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62aba5d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4163100 con 0x7f62a4071f00 2024-05-27T00:39:41.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62aba5d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4162f80 con 0x7f62a40b3f10 2024-05-27T00:39:41.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62aba5d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f62a414ba80 con 0x7f62a41637e0 2024-05-27T00:39:41.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f9a35dc7640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a300ce670 0x7f9a301f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62a97f9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4071f00 0x7f62a41378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62a97f9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4071f00 0x7f62a41378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50800/0 (socket says 172.21.15.111:50800) 2024-05-27T00:39:41.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f62a97f9640 1 -- 172.21.15.111:0/1381398314 learned_addr learned my addr 172.21.15.111:0/1381398314 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f06118a7640 1 Processor -- start 2024-05-27T00:39:41.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f06118a7640 1 -- start start 2024-05-27T00:39:41.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f378d7fa640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f377006be80 0x7f377006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3006cc30 con 0x7f9a300b5600 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3006cf30 con 0x7f9a300b5220 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe360d72640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe35c154d00 0x7fe35c142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7fe35bfff640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c06dfc0 0x7fe35c141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe360d72640 1 -- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c06dfc0 msgr2=0x7fe35c141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe360d72640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe35c06dfc0 0x7fe35c141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe360d72640 1 -- 172.21.15.111:0/2295367975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe35c15bed0 msgr2=0x7fe35c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe360d72640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe35c15bed0 0x7fe35c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe360d72640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c1476a0 con 0x7fe35c154d00 2024-05-27T00:39:41.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe35b7fe640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe35c15bed0 0x7fe35c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:41.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f378ffff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f377006be80 0x7f377006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f37780ab670 con 0x7f379014c020 2024-05-27T00:39:41.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.110+0000 7f2f0ad9d640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f040e97d0 0x7f2f04036b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe360d72640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe35c154d00 0x7fe35c142330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe354002220 tx=0x7fe35400cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f2f0a59c640 1 -- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040e9350 msgr2=0x7f2f040365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2f040e9350 0x7f2f040365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f2f0a59c640 1 -- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f040e97d0 msgr2=0x7f2f04036b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f040e97d0 0x7f2f04036b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f2f0a59c640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0417da80 con 0x7f2ef40039e0 2024-05-27T00:39:41.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f378d7fa640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] conn(0x7f3770075580 0x7f37700779c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f9a07aba640 1 -- start start 2024-05-27T00:39:41.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354003b70 con 0x7fe35c154d00 2024-05-27T00:39:41.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe354003d30 con 0x7fe35c154d00 2024-05-27T00:39:41.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354015ba0 con 0x7fe35c154d00 2024-05-27T00:39:41.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f62a97f9640 1 -- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62a41637e0 msgr2=0x7f62a4165e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:39:41.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f62a97f9640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f62a41637e0 0x7f62a4165e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f62a97f9640 1 -- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62a40b3f10 msgr2=0x7f62a41397f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:39:41.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f62a97f9640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62a40b3f10 0x7f62a41397f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f62a97f9640 1 -- 172.21.15.111:0/1381398314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a41663c0 con 0x7f62a4071f00 2024-05-27T00:39:41.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f3794c57640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] conn(0x7f3770075580 0x7f37700779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f2f0a59c640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef40039e0 0x7f2f0404e7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c065130 tx=0x7f2f0c06a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c1473c0 con 0x7fe35c154d00 2024-05-27T00:39:41.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f9a07aba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f9a000ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c140a20 con 0x7fe35c154d00 2024-05-27T00:39:41.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f773c01a070 con 0x7f7740006a80 2024-05-27T00:39:41.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.111+0000 7f774fcb3640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7740005810 con 0x7f7740006a80 2024-05-27T00:39:41.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f773a7fc640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f772c06be80 0x7f772c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f62a97f9640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4071f00 0x7f62a41378b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f629402a790 tx=0x7f62940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06118a7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0604006ea0 0x7f06040471d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06118a7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f060404ffd0 0x7f060404e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f3794c57640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] conn(0x7f3770075580 0x7f37700779c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 --> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3778047e50 con 0x7f3770075580 2024-05-27T00:39:41.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06118a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040deaf0 0x7f06040def80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06108a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0604006ea0 0x7f06040471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f774d24e640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f772c06be80 0x7f772c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f378ffff640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f377006be80 0x7f377006e340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3780009f30 tx=0x7f3780009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37780b2050 con 0x7f379014c020 2024-05-27T00:39:41.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f773c056590 con 0x7f7740006a80 2024-05-27T00:39:41.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c068b00 con 0x7f2ef40039e0 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c04fcd0 con 0x7f2ef40039e0 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c04ffb0 con 0x7f2ef40039e0 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f773a7fc640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] conn(0x7f772c076250 0x7f772c0786f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f772c078dc0 con 0x7f772c076250 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f773c054f20 con 0x7f7740006a80 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a07aba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00158810 0x7f9a001579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a07aba640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a0015bd90 0x7f9a000cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a07aba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9a000d3990 con 0x7f99ec004800 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f2f0b59e640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0404dcd0 con 0x7f2ef40039e0 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f060bfff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f060404ffd0 0x7f060404e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f060bfff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f060404ffd0 0x7f060404e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47726/0 (socket says 172.21.15.111:47726) 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06108a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0604006ea0 0x7f06040471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56294/0 (socket says 172.21.15.111:56294) 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f060bfff640 1 -- 172.21.15.111:0/1945164792 learned_addr learned my addr 172.21.15.111:0/1945164792 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06110a6640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040deaf0 0x7f06040def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0604054940 con 0x7f06040deaf0 2024-05-27T00:39:41.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f06040547c0 con 0x7f0604006ea0 2024-05-27T00:39:41.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f060404f550 con 0x7f060404ffd0 2024-05-27T00:39:41.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294040020 con 0x7f62a4071f00 2024-05-27T00:39:41.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.112+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a4166670 con 0x7f62a4071f00 2024-05-27T00:39:41.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a41370b0 con 0x7f62a4071f00 2024-05-27T00:39:41.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f2f0b59e640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0417cdc0 con 0x7f2ef40039e0 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe35c068680 con 0x7fe35c154d00 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a05856640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f9a000ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a05055640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00158810 0x7f9a001579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a05856640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f9a000ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50814/0 (socket says 172.21.15.111:50814) 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a05856640 1 -- 172.21.15.111:0/3234660245 learned_addr learned my addr 172.21.15.111:0/3234660245 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a06057640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a0015bd90 0x7f9a000cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a07aba640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0006c200 con 0x7f9a00158810 2024-05-27T00:39:41.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f9a07aba640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9a000d3b10 con 0x7f9a0015bd90 2024-05-27T00:39:41.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f774d24e640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f772c06be80 0x7f772c06e340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7744034000 tx=0x7f774402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a06057640 1 -- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00158810 msgr2=0x7f9a001579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a06057640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a00158810 0x7f9a001579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a06057640 1 -- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 msgr2=0x7f9a000ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== osd.2 v2:172.21.15.111:6808/1204898981 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3778047e50 con 0x7f3770075580 2024-05-27T00:39:41.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f774e250640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] conn(0x7f772c076250 0x7f772c0786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3540050d0 con 0x7fe35c154d00 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f060bfff640 1 -- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0604006ea0 msgr2=0x7f06040471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f060bfff640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0604006ea0 0x7f06040471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f060bfff640 1 -- 172.21.15.111:0/1945164792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040deaf0 msgr2=0x7f06040def80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f060bfff640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040deaf0 0x7f06040def80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.113+0000 7f060bfff640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06040438d0 con 0x7f060404ffd0 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f06110a6640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06040deaf0 0x7f06040def80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:41.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62a4068460 con 0x7f62a4071f00 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a06057640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f9a000ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a06057640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a00158d80 con 0x7f9a0015bd90 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a05856640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ec004800 0x7f9a000ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a34dc5640 1 -- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 msgr2=0x7f9a300af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a34dc5640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a300b5220 0x7f9a300af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a34dc5640 1 -- 172.21.15.111:0/2660685201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a300ce670 msgr2=0x7f9a301f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a34dc5640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a300ce670 0x7f9a301f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a34dc5640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a301f6a90 con 0x7f9a300b5600 2024-05-27T00:39:41.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f060bfff640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f060404ffd0 0x7f060404e180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c440 tx=0x7f05fc00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f774e250640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] conn(0x7f772c076250 0x7f772c0786f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6294038040 con 0x7f62a4071f00 2024-05-27T00:39:41.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294040020 con 0x7f62a4071f00 2024-05-27T00:39:41.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f9a06057640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a0015bd90 0x7f9a000cd500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a000d1c20 tx=0x7f99f800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.114+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f0417dd60 con 0x7f2ef40039e0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc017070 con 0x7f060404ffd0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc005ce0 con 0x7f060404ffd0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0072b0 con 0x7f060404ffd0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0604043bb0 con 0x7f060404ffd0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0604044170 con 0x7f060404ffd0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f9a34dc5640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a300afb50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00b470 tx=0x7f9a2c00b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c015020 con 0x7f9a300b5600 2024-05-27T00:39:41.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c011890 con 0x7f9a300b5600 2024-05-27T00:39:41.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7fe358ff9640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe34806bf30 0x7fe34806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe3540932a0 con 0x7fe35c154d00 2024-05-27T00:39:41.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7fe358ff9640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] conn(0x7fe348075620 0x7fe348077a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe354024e80 con 0x7fe348075620 2024-05-27T00:39:41.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8008780 con 0x7f9a0015bd90 2024-05-27T00:39:41.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== osd.1 v2:172.21.15.111:6800/2862511107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f772c078dc0 con 0x7f772c076250 2024-05-27T00:39:41.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8008920 con 0x7f9a0015bd90 2024-05-27T00:39:41.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0604005390 con 0x7f060404ffd0 2024-05-27T00:39:41.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f9a07aba640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a00159060 con 0x7f9a0015bd90 2024-05-27T00:39:41.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f9a07aba640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a00159620 con 0x7f9a0015bd90 2024-05-27T00:39:41.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a301f6d70 con 0x7f9a300b5600 2024-05-27T00:39:41.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.115+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c015020 con 0x7f9a300b5600 2024-05-27T00:39:41.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a301f7330 con 0x7f9a300b5600 2024-05-27T00:39:41.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe35409a050 con 0x7fe35c154d00 2024-05-27T00:39:41.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7fe35b7fe640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] conn(0x7fe348075620 0x7fe348077a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7fe35bfff640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe34806bf30 0x7fe34806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a2c018020 con 0x7f9a300b5600 2024-05-27T00:39:41.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a30068b00 con 0x7f9a300b5600 2024-05-27T00:39:41.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7fe35bfff640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe34806bf30 0x7fe34806e3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe350000f60 tx=0x7fe350007c60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6294043020 con 0x7f62a4071f00 2024-05-27T00:39:41.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f6299ffb640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f628806c030 0x7f628806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6294046e00 con 0x7f62a4071f00 2024-05-27T00:39:41.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05fc01a070 con 0x7f060404ffd0 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8015670 con 0x7f9a0015bd90 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.116+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f0c0ac030 con 0x7f2ef40039e0 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7f2efaffd640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2eec06beb0 0x7f2eec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2f0c0f2460 con 0x7f2ef40039e0 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7f2efaffd640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] conn(0x7f2eec0755b0 0x7f2eec0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.117+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f0c0b5e50 con 0x7f2eec0755b0 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f0c0f0660 con 0x7f2ef40039e0 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f2f09d9b640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2eec06beb0 0x7f2eec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f2f0ad9d640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] conn(0x7f2eec0755b0 0x7f2eec0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f6299ffb640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] conn(0x7f6288076380 0x7f6288078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 --> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6288078ef0 con 0x7f6288076380 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f62940761b0 con 0x7f62a4071f00 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.118+0000 7f9a25ffb640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9a1406c030 0x7f9a1406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9a2c08ab90 con 0x7f9a300b5600 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f9a25ffb640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] conn(0x7f9a140756a0 0x7f9a14077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a140781b0 con 0x7f9a140756a0 2024-05-27T00:39:41.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f06097fa640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f05f806bf00 0x7f05f806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a2c054dc0 con 0x7f9a300b5600 2024-05-27T00:39:41.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f2f0ad9d640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] conn(0x7f2eec0755b0 0x7f2eec0779f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f9a355c6640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9a1406c030 0x7f9a1406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f62a8ff8640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f628806c030 0x7f628806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a00068680 con 0x7f9a0015bd90 2024-05-27T00:39:41.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7fe35b7fe640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] conn(0x7fe348075620 0x7fe348077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99f801c020 con 0x7f9a0015bd90 2024-05-27T00:39:41.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f9a35dc7640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] conn(0x7f9a140756a0 0x7f9a14077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f62a9ffa640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] conn(0x7f6288076380 0x7f6288078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f06108a5640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f05f806bf00 0x7f05f806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f2f09d9b640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2eec06beb0 0x7f2eec06e370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f040371b0 tx=0x7f2f00009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f05fc08a4e0 con 0x7f060404ffd0 2024-05-27T00:39:41.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f06097fa640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] conn(0x7f05f8075680 0x7f05f8077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05f8078190 con 0x7f05f8075680 2024-05-27T00:39:41.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.119+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05fc091050 con 0x7f060404ffd0 2024-05-27T00:39:41.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f99f67fc640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f99e406bf00 0x7f99e406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f9a355c6640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9a1406c030 0x7f9a1406e4f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a28011340 tx=0x7f9a2800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f62a9ffa640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] conn(0x7f6288076380 0x7f6288078820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99f808b500 con 0x7f9a0015bd90 2024-05-27T00:39:41.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f9a35dc7640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] conn(0x7f9a140756a0 0x7f9a14077ae0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f62a8ff8640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f628806c030 0x7f628806e4f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62a0000f60 tx=0x7f62a00079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f06108a5640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f05f806bf00 0x7f05f806e3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0600034000 tx=0x7f060002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f9a05856640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f99e406bf00 0x7f99e406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f99f67fc640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] conn(0x7f99e4075680 0x7f99e4077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99f8008ad0 con 0x7f99e4075680 2024-05-27T00:39:41.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f06110a6640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] conn(0x7f05f8075680 0x7f05f8077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99f8092050 con 0x7f9a0015bd90 2024-05-27T00:39:41.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== osd.4 v2:172.21.15.153:6808/3644921955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe354024e80 con 0x7fe348075620 2024-05-27T00:39:41.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f9a05055640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] conn(0x7f99e4075680 0x7f99e4077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.120+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== osd.0 v2:172.21.15.111:6816/4118349595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f0c0b5e50 con 0x7f2eec0755b0 2024-05-27T00:39:41.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.121+0000 7f06110a6640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] conn(0x7f05f8075680 0x7f05f8077ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.121+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== osd.5 v2:172.21.15.153:6800/668441948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6288078ef0 con 0x7f6288076380 2024-05-27T00:39:41.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.121+0000 7f9a05856640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f99e406bf00 0x7f99e406e3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a000732a0 tx=0x7f99fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.121+0000 7f9a05055640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] conn(0x7f99e4075680 0x7f99e4077ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.121+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== osd.6 v2:172.21.15.153:6824/3277363095 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05f8078190 con 0x7f05f8075680 2024-05-27T00:39:41.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.121+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== osd.7 v2:172.21.15.153:6816/2433121395 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a140781b0 con 0x7f9a140756a0 2024-05-27T00:39:41.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.122+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== osd.3 v2:172.21.15.111:6817/1957218503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99f8008ad0 con 0x7f99e4075680 2024-05-27T00:39:41.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.145+0000 7f3795458640 1 -- 172.21.15.111:0/71825070 --> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3774002dd0 con 0x7f3770075580 2024-05-27T00:39:41.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.146+0000 7f378d7fa640 1 -- 172.21.15.111:0/71825070 <== osd.2 v2:172.21.15.111:6808/1204898981 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3774002dd0 con 0x7f3770075580 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.146+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] conn(0x7f3770075580 msgr2=0x7f37700779c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.146+0000 7f37867fc640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:6808/1204898981,v1:172.21.15.111:6809/1204898981] conn(0x7f3770075580 0x7f37700779c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f377006be80 msgr2=0x7f377006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f37867fc640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f377006be80 0x7f377006e340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3780009f30 tx=0x7f3780009190 comp rx=0 tx=0).stop 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379014c020 msgr2=0x7f3790149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f37867fc640 1 --2- 172.21.15.111:0/71825070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379014c020 0x7f3790149370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f377802a1a0 tx=0x7f37780046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f3794c57640 1 -- 172.21.15.111:0/71825070 reap_dead start 2024-05-27T00:39:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 shutdown_connections 2024-05-27T00:39:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.147+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 >> 172.21.15.111:0/71825070 conn(0x7f379006d300 msgr2=0x7f37900b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.148+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 shutdown_connections 2024-05-27T00:39:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.148+0000 7f37867fc640 1 -- 172.21.15.111:0/71825070 wait complete. 2024-05-27T00:39:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.150+0000 7f774fcb3640 1 -- 172.21.15.111:0/371951923 --> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f774000b330 con 0x7f772c076250 2024-05-27T00:39:41.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.151+0000 7f773a7fc640 1 -- 172.21.15.111:0/371951923 <== osd.1 v2:172.21.15.111:6800/2862511107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f774000b330 con 0x7f772c076250 2024-05-27T00:39:41.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.151+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 --> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe35c15e810 con 0x7fe348075620 2024-05-27T00:39:41.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 --> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f04005390 con 0x7f2eec0755b0 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] conn(0x7f772c076250 msgr2=0x7f772c0786f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.111:6800/2862511107,v1:172.21.15.111:6801/2862511107] conn(0x7f772c076250 0x7f772c0786f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f772c06be80 msgr2=0x7f772c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f772c06be80 0x7f772c06e340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7744034000 tx=0x7f774402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740006a80 msgr2=0x7f774004e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 --2- 172.21.15.111:0/371951923 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7740006a80 0x7f774004e1d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c3f0 tx=0x7f773c00c8b0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f774e250640 1 -- 172.21.15.111:0/371951923 reap_dead start 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 shutdown_connections 2024-05-27T00:39:41.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 >> 172.21.15.111:0/371951923 conn(0x7f774000a430 msgr2=0x7f774000a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f2efaffd640 1 -- 172.21.15.111:0/1615949821 <== osd.0 v2:172.21.15.111:6816/4118349595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f04005390 con 0x7f2eec0755b0 2024-05-27T00:39:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f06118a7640 1 -- 172.21.15.111:0/1945164792 --> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0604048710 con 0x7f05f8075680 2024-05-27T00:39:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.152+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 shutdown_connections 2024-05-27T00:39:41.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f772bfff640 1 -- 172.21.15.111:0/371951923 wait complete. 2024-05-27T00:39:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7fe358ff9640 1 -- 172.21.15.111:0/2295367975 <== osd.4 v2:172.21.15.153:6808/3644921955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe35c15e810 con 0x7fe348075620 2024-05-27T00:39:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] conn(0x7f2eec0755b0 msgr2=0x7f2eec0779f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.111:6816/4118349595,v1:172.21.15.111:6818/4118349595] conn(0x7f2eec0755b0 0x7f2eec0779f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2eec06beb0 msgr2=0x7f2eec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2eec06beb0 0x7f2eec06e370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f040371b0 tx=0x7f2f00009040 comp rx=0 tx=0).stop 2024-05-27T00:39:41.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef40039e0 msgr2=0x7f2f0404e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 --2- 172.21.15.111:0/1615949821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ef40039e0 0x7f2f0404e7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c065130 tx=0x7f2f0c06a0a0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f06097fa640 1 -- 172.21.15.111:0/1945164792 <== osd.6 v2:172.21.15.153:6824/3277363095 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0604048710 con 0x7f05f8075680 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2f0ad9d640 1 -- 172.21.15.111:0/1615949821 reap_dead start 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 shutdown_connections 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 >> 172.21.15.111:0/1615949821 conn(0x7f2f04009fb0 msgr2=0x7f2f04033ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 shutdown_connections 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7f2ef8ff9640 1 -- 172.21.15.111:0/1615949821 wait complete. 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.153+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] conn(0x7fe348075620 msgr2=0x7fe348077a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6808/3644921955,v1:172.21.15.153:6809/3644921955] conn(0x7fe348075620 0x7fe348077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 --> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62a40b34e0 con 0x7f6288076380 2024-05-27T00:39:41.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe34806bf30 msgr2=0x7fe34806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe34806bf30 0x7fe34806e3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe350000f60 tx=0x7fe350007c60 comp rx=0 tx=0).stop 2024-05-27T00:39:41.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe35c154d00 msgr2=0x7fe35c142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 --2- 172.21.15.111:0/2295367975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe35c154d00 0x7fe35c142330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe354002220 tx=0x7fe35400cb10 comp rx=0 tx=0).stop 2024-05-27T00:39:41.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe360d72640 1 -- 172.21.15.111:0/2295367975 reap_dead start 2024-05-27T00:39:41.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 --> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a3015cfb0 con 0x7f9a140756a0 2024-05-27T00:39:41.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 shutdown_connections 2024-05-27T00:39:41.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 >> 172.21.15.111:0/2295367975 conn(0x7fe35c073130 msgr2=0x7fe35c0ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 --> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a00068680 con 0x7f99e4075680 2024-05-27T00:39:41.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f6299ffb640 1 -- 172.21.15.111:0/1381398314 <== osd.5 v2:172.21.15.153:6800/668441948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62a40b34e0 con 0x7f6288076380 2024-05-27T00:39:41.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 shutdown_connections 2024-05-27T00:39:41.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7fe3627d5640 1 -- 172.21.15.111:0/2295367975 wait complete. 2024-05-27T00:39:41.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f9a25ffb640 1 -- 172.21.15.111:0/2660685201 <== osd.7 v2:172.21.15.153:6816/2433121395 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9a3015cfb0 con 0x7f9a140756a0 2024-05-27T00:39:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] conn(0x7f05f8075680 msgr2=0x7f05f8077ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.154+0000 7f05f2ffd640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6824/3277363095,v1:172.21.15.153:6825/3277363095] conn(0x7f05f8075680 0x7f05f8077ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f05f806bf00 msgr2=0x7f05f806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f05f2ffd640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f05f806bf00 0x7f05f806e3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0600034000 tx=0x7f060002e000 comp rx=0 tx=0).stop 2024-05-27T00:39:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f060404ffd0 msgr2=0x7f060404e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f05f2ffd640 1 --2- 172.21.15.111:0/1945164792 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f060404ffd0 0x7f060404e180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c440 tx=0x7f05fc00c900 comp rx=0 tx=0).stop 2024-05-27T00:39:41.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f99f67fc640 1 -- 172.21.15.111:0/3234660245 <== osd.3 v2:172.21.15.111:6817/1957218503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9a00068680 con 0x7f99e4075680 2024-05-27T00:39:41.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f06110a6640 1 -- 172.21.15.111:0/1945164792 reap_dead start 2024-05-27T00:39:41.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] conn(0x7f6288076380 msgr2=0x7f6288078820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f62aba5d640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6800/668441948,v1:172.21.15.153:6801/668441948] conn(0x7f6288076380 0x7f6288078820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 shutdown_connections 2024-05-27T00:39:41.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 >> 172.21.15.111:0/1945164792 conn(0x7f0604009fb0 msgr2=0x7f0604036430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.155+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] conn(0x7f9a140756a0 msgr2=0x7f9a14077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6816/2433121395,v1:172.21.15.153:6817/2433121395] conn(0x7f9a140756a0 0x7f9a14077ae0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] conn(0x7f99e4075680 msgr2=0x7f99e4077ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f628806c030 msgr2=0x7f628806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f628806c030 0x7f628806e4f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62a0000f60 tx=0x7f62a00079b0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4071f00 msgr2=0x7f62a41378b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 --2- 172.21.15.111:0/1381398314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4071f00 0x7f62a41378b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f629402a790 tx=0x7f62940046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.111:6817/1957218503,v1:172.21.15.111:6819/1957218503] conn(0x7f99e4075680 0x7f99e4077ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62a9ffa640 1 -- 172.21.15.111:0/1381398314 reap_dead start 2024-05-27T00:39:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 shutdown_connections 2024-05-27T00:39:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 shutdown_connections 2024-05-27T00:39:41.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 >> 172.21.15.111:0/1381398314 conn(0x7f62a406b390 msgr2=0x7f62a40b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f05f2ffd640 1 -- 172.21.15.111:0/1945164792 wait complete. 2024-05-27T00:39:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9a1406c030 msgr2=0x7f9a1406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9a1406c030 0x7f9a1406e4f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a28011340 tx=0x7f9a2800b040 comp rx=0 tx=0).stop 2024-05-27T00:39:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 msgr2=0x7f9a300afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 --2- 172.21.15.111:0/2660685201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a300b5600 0x7f9a300afb50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00b470 tx=0x7f9a2c00b930 comp rx=0 tx=0).stop 2024-05-27T00:39:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f99e406bf00 msgr2=0x7f99e406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f99e406bf00 0x7f99e406e3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9a000732a0 tx=0x7f99fc009040 comp rx=0 tx=0).stop 2024-05-27T00:39:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a35dc7640 1 -- 172.21.15.111:0/2660685201 reap_dead start 2024-05-27T00:39:41.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a0015bd90 msgr2=0x7f9a000cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 --2- 172.21.15.111:0/3234660245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9a0015bd90 0x7f9a000cd500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a000d1c20 tx=0x7f99f800eef0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 shutdown_connections 2024-05-27T00:39:41.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f62aba5d640 1 -- 172.21.15.111:0/1381398314 wait complete. 2024-05-27T00:39:41.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a06057640 1 -- 172.21.15.111:0/3234660245 reap_dead start 2024-05-27T00:39:41.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 shutdown_connections 2024-05-27T00:39:41.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 >> 172.21.15.111:0/3234660245 conn(0x7f9a000d4570 msgr2=0x7f9a000b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 shutdown_connections 2024-05-27T00:39:41.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.156+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 >> 172.21.15.111:0/2660685201 conn(0x7f9a300723e0 msgr2=0x7f9a300b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.157+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 shutdown_connections 2024-05-27T00:39:41.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.157+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 shutdown_connections 2024-05-27T00:39:41.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.157+0000 7f9a365c8640 1 -- 172.21.15.111:0/2660685201 wait complete. 2024-05-27T00:39:41.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.157+0000 7f99ebfff640 1 -- 172.21.15.111:0/3234660245 wait complete. 2024-05-27T00:39:41.228 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-27T00:39:41.229 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-27T00:39:41.230 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.231 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-27T00:39:41.231 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.232 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-27T00:39:41.232 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.233 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-27T00:39:41.233 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.234 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-27T00:39:41.234 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-27T00:39:41.236 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:41.236 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-27T00:39:41.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.439+0000 7fe11a03c640 1 Processor -- start 2024-05-27T00:39:41.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.439+0000 7fe11a03c640 1 -- start start 2024-05-27T00:39:41.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7ff2d8363640 1 Processor -- start 2024-05-27T00:39:41.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7ff39d89e640 1 Processor -- start 2024-05-27T00:39:41.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.439+0000 7fc71e3a5640 1 Processor -- start 2024-05-27T00:39:41.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7fc71e3a5640 1 -- start start 2024-05-27T00:39:41.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7f09357a5640 1 Processor -- start 2024-05-27T00:39:41.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7f5011cd2640 1 Processor -- start 2024-05-27T00:39:41.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7ff2d8363640 1 -- start start 2024-05-27T00:39:41.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7f5011cd2640 1 -- start start 2024-05-27T00:39:41.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7ff39d89e640 1 -- start start 2024-05-27T00:39:41.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7f6a56a7f640 1 Processor -- start 2024-05-27T00:39:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7f6a56a7f640 1 -- start start 2024-05-27T00:39:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11a03c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 0x7fe11406c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11a03c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe1140b3420 0x7fe1140cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11a03c640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe11415e590 con 0x7fe1140cfb50 2024-05-27T00:39:41.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11a03c640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fe1140d24c0 con 0x7fe1140b3420 2024-05-27T00:39:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11a03c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe1140b3880 con 0x7fe11415e1b0 2024-05-27T00:39:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff39d89e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff3980d0ce0 0x7ff39806b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff39d89e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff39806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff39d89e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39815afc0 con 0x7ff398068b00 2024-05-27T00:39:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11903a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 0x7fe11406c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11903a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 0x7fe11406c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47758/0 (socket says 172.21.15.111:47758) 2024-05-27T00:39:41.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe11903a640 1 -- 172.21.15.111:0/3157958232 learned_addr learned my addr 172.21.15.111:0/3157958232 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff2d8363640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff2d015e240 0x7ff2d00b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff2d8363640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d0162340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff2d8363640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2d00d21c0 con 0x7ff2d006bff0 2024-05-27T00:39:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff2d8363640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d0162d40 con 0x7ff2d015e240 2024-05-27T00:39:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff2d8363640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d00d24a0 con 0x7ff2d0161ec0 2024-05-27T00:39:41.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.440+0000 7f09357a5640 1 -- start start 2024-05-27T00:39:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7f09357a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f09300af740 0x7f093006b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff39d89e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff39815b2a0 con 0x7ff3980d0ce0 2024-05-27T00:39:41.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7ff39d89e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff39815b580 con 0x7ff3980af4d0 2024-05-27T00:39:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d0162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f5011cd2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c0b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fe118839640 1 --2- 172.21.15.111:0/3157958232 >> v2:172.21.15.153:3300/0 conn(0x7fe1140b3420 0x7fe1140cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f09357a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093006d6f0 0x7f093006db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d0162340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47772/0 (socket says 172.21.15.111:47772) 2024-05-27T00:39:41.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 -- 172.21.15.111:0/3655999005 learned_addr learned my addr 172.21.15.111:0/3655999005 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f5011cd2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f500c0d21c0 con 0x7f500c161ec0 2024-05-27T00:39:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f5011cd2640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f500c0d24a0 con 0x7f500c06c700 2024-05-27T00:39:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f5011cd2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f500c162d40 con 0x7f500c15e240 2024-05-27T00:39:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f09357a5640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093015ace0 con 0x7f09300d0ce0 2024-05-27T00:39:41.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c0b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fbcbff5c640 1 Processor -- start 2024-05-27T00:39:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c0b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47776/0 (socket says 172.21.15.111:47776) 2024-05-27T00:39:41.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f500b7fe640 1 -- 172.21.15.111:0/3685103813 learned_addr learned my addr 172.21.15.111:0/3685103813 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d60ff640 1 --2- 172.21.15.111:0/3655999005 >> v2:172.21.15.153:3300/0 conn(0x7ff2d015e240 0x7ff2d00b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f0927fff640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f09300d0ce0 0x7f093006c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47260/0 (socket says 172.21.15.111:47260) 2024-05-27T00:39:41.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fbcbff5c640 1 -- start start 2024-05-27T00:39:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f500b7fe640 1 -- 172.21.15.111:0/3685103813 >> v1:172.21.15.153:6789/0 conn(0x7f500c06c700 legacy=0x7f500c0d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f0927fff640 1 -- 172.21.15.111:0/2626641868 learned_addr learned my addr 172.21.15.111:0/2626641868 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f092effd640 1 --2- 172.21.15.111:0/2626641868 >> v2:172.21.15.111:3300/0 conn(0x7f09300af740 0x7f093006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f092f7fe640 1 --2- 172.21.15.111:0/2626641868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093006d6f0 0x7f093006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f093006e080 con 0x7f09300af740 2024-05-27T00:39:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f093015afc0 con 0x7f093006d6f0 2024-05-27T00:39:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 -- 172.21.15.111:0/3655999005 >> v2:172.21.15.153:3300/0 conn(0x7ff2d015e240 msgr2=0x7ff2d00b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a56a7f640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a480e59b0 0x7f6a480e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 --2- 172.21.15.111:0/3655999005 >> v2:172.21.15.153:3300/0 conn(0x7ff2d015e240 0x7ff2d00b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 -- 172.21.15.111:0/3655999005 >> v1:172.21.15.111:6789/0 conn(0x7ff2d006bff0 legacy=0x7ff2d006c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:41.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a56a7f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6a4800a520 0x7f6a480e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a56a7f640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a48051c90 con 0x7f6a48051720 2024-05-27T00:39:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a56a7f640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f6a48051f70 con 0x7f6a4800a520 2024-05-27T00:39:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a56a7f640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6a48052250 con 0x7f6a480e59b0 2024-05-27T00:39:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f500b7fe640 1 -- 172.21.15.111:0/3685103813 >> v1:172.21.15.111:6789/0 conn(0x7f500c161ec0 legacy=0x7f500c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fe10bfff640 1 -- 172.21.15.111:0/3157958232 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075399664 0 0) 0x7fe11415e590 con 0x7fe1140cfb50 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fe10bfff640 1 -- 172.21.15.111:0/3157958232 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe100003520 con 0x7fe1140cfb50 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fe11903a640 1 -- 172.21.15.111:0/3157958232 >> v2:172.21.15.153:3300/0 conn(0x7fe1140b3420 msgr2=0x7fe1140cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fe11903a640 1 --2- 172.21.15.111:0/3157958232 >> v2:172.21.15.153:3300/0 conn(0x7fe1140b3420 0x7fe1140cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fe11903a640 1 -- 172.21.15.111:0/3157958232 >> v1:172.21.15.153:6789/0 conn(0x7fe1140cfb50 legacy=0x7fe114161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fe11903a640 1 -- 172.21.15.111:0/3157958232 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11406c6d0 con 0x7fe11415e1b0 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f092f7fe640 1 -- 172.21.15.111:0/2626641868 >> v2:172.21.15.111:3300/0 conn(0x7f09300af740 msgr2=0x7f093006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f092f7fe640 1 --2- 172.21.15.111:0/2626641868 >> v2:172.21.15.111:3300/0 conn(0x7f09300af740 0x7f093006b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f092f7fe640 1 -- 172.21.15.111:0/2626641868 >> v1:172.21.15.153:6789/0 conn(0x7f09300d0ce0 legacy=0x7f093006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f092f7fe640 1 -- 172.21.15.111:0/2626641868 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093015b2a0 con 0x7f093006d6f0 2024-05-27T00:39:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7ff2d58fe640 1 -- 172.21.15.111:0/3655999005 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d00d2780 con 0x7ff2d0161ec0 2024-05-27T00:39:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7ff2d60ff640 1 --2- 172.21.15.111:0/3655999005 >> v2:172.21.15.153:3300/0 conn(0x7ff2d015e240 0x7ff2d00b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7fe11903a640 1 --2- 172.21.15.111:0/3157958232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 0x7fe11406c040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1140d2640 tx=0x7fe104032aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40981a3f01671c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc71e3a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc71806b390 0x7fc7180b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc717fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc71806b390 0x7fc7180b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc717fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc71806b390 0x7fc7180b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50838/0 (socket says 172.21.15.111:50838) 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc717fff640 1 -- 172.21.15.111:0/554809971 learned_addr learned my addr 172.21.15.111:0/554809971 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/554809971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc718163cb0 0x7fc71816c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7180b42b0 con 0x7fc7180b3c80 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc7180b4590 con 0x7fc71806b390 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.441+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc7180b4870 con 0x7fc718163cb0 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fc717fff640 1 -- 172.21.15.111:0/554809971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc718163cb0 msgr2=0x7fc71816c2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fc717fff640 1 --2- 172.21.15.111:0/554809971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc718163cb0 0x7fc71816c2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fc717fff640 1 -- 172.21.15.111:0/554809971 >> v1:172.21.15.111:6789/0 conn(0x7fc7180b3c80 legacy=0x7fc71806b770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7fc717fff640 1 -- 172.21.15.111:0/554809971 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7180b4b50 con 0x7fc71806b390 2024-05-27T00:39:41.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7fc717fff640 1 --2- 172.21.15.111:0/554809971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71806b390 0x7fc7180b3740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009ad0 tx=0x7fc70c02a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1267bfaee9144a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a55a7d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a480e59b0 0x7f6a480e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a55a7d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6a480e59b0 0x7f6a480e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56328/0 (socket says 172.21.15.111:56328) 2024-05-27T00:39:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7f6a55a7d640 1 -- 172.21.15.111:0/645230573 learned_addr learned my addr 172.21.15.111:0/645230573 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.443+0000 7ff2d58fe640 1 --2- 172.21.15.111:0/3655999005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d0162340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009870 tx=0x7ff2c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f29a1dcafdefda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff2d50fd640 1 -- 172.21.15.111:0/3655999005 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8037070 con 0x7ff2d0161ec0 2024-05-27T00:39:41.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff2d50fd640 1 -- 172.21.15.111:0/3655999005 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c8032440 con 0x7ff2d0161ec0 2024-05-27T00:39:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.442+0000 7f500b7fe640 1 -- 172.21.15.111:0/3685103813 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c0d2780 con 0x7f500c15e240 2024-05-27T00:39:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fe10bfff640 1 -- 172.21.15.111:0/3157958232 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104036860 con 0x7fe11415e1b0 2024-05-27T00:39:41.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbff5c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fe10bfff640 1 -- 172.21.15.111:0/3157958232 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe104036a00 con 0x7fe11415e1b0 2024-05-27T00:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fe10bfff640 1 -- 172.21.15.111:0/3157958232 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104036e10 con 0x7fe11415e1b0 2024-05-27T00:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbff5c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb806d3a0 con 0x7fbcb8072190 2024-05-27T00:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbff5c640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb80bc1d0 con 0x7fbcb814bec0 2024-05-27T00:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbff5c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb80d6710 con 0x7fbcb80d5d70 2024-05-27T00:39:41.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff2d50fd640 1 -- 172.21.15.111:0/3655999005 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8032810 con 0x7ff2d0161ec0 2024-05-27T00:39:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fc716ffd640 1 -- 172.21.15.111:0/554809971 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c036070 con 0x7fc71806b390 2024-05-27T00:39:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fc716ffd640 1 -- 172.21.15.111:0/554809971 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70c031600 con 0x7fc71806b390 2024-05-27T00:39:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fc716ffd640 1 -- 172.21.15.111:0/554809971 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0319d0 con 0x7fc71806b390 2024-05-27T00:39:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a54a7b640 1 -- 172.21.15.111:0/645230573 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440466674 0 0) 0x7f6a48051c90 con 0x7f6a48051720 2024-05-27T00:39:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a54a7b640 1 -- 172.21.15.111:0/645230573 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a3c003520 con 0x7f6a48051720 2024-05-27T00:39:41.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a5627e640 1 --2- 172.21.15.111:0/645230573 >> v2:172.21.15.111:3300/0 conn(0x7f6a4800a520 0x7f6a480e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f500b7fe640 1 --2- 172.21.15.111:0/3685103813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c0b7310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098a0 tx=0x7f4ffc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd7372e0174469f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff39806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff39806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47796/0 (socket says 172.21.15.111:47796) 2024-05-27T00:39:41.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 -- 172.21.15.111:0/3858348432 learned_addr learned my addr 172.21.15.111:0/3858348432 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbdcf8640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbcb814bec0 0x7fbcb806de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47302/0 (socket says 172.21.15.111:47302) 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbe4f9640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbcb8072190 0x7fbcb80bb5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36672/0 (socket says 172.21.15.111:36672) 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7fbcbdcf8640 1 -- 172.21.15.111:0/2806718458 learned_addr learned my addr 172.21.15.111:0/2806718458 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 -- 172.21.15.111:0/3157958232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 msgr2=0x7fe11406c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 --2- 172.21.15.111:0/3157958232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 0x7fe11406c040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1140d2640 tx=0x7fe104032aa0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 -- 172.21.15.111:0/3157958232 shutdown_connections 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 --2- 172.21.15.111:0/3157958232 >> v2:172.21.15.153:3300/0 conn(0x7fe1140b3420 0x7fe1140cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 --2- 172.21.15.111:0/3157958232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe11415e1b0 0x7fe11406c040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 -- 172.21.15.111:0/3157958232 >> 172.21.15.111:0/3157958232 conn(0x7fe1140d4450 msgr2=0x7fe1140d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 -- 172.21.15.111:0/3157958232 shutdown_connections 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fe11a03c640 1 -- 172.21.15.111:0/3157958232 wait complete. 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71806b390 msgr2=0x7fc7180b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/554809971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71806b390 0x7fc7180b3740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009ad0 tx=0x7fc70c02a490 comp rx=0 tx=0).stop 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f500affd640 1 -- 172.21.15.111:0/3685103813 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc037070 con 0x7f500c15e240 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f500affd640 1 -- 172.21.15.111:0/3685103813 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ffc032440 con 0x7f500c15e240 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7f5011cd2640 1 -- 172.21.15.111:0/3685103813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 msgr2=0x7f500c0b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3685103813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c0b7310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0098a0 tx=0x7f4ffc02a210 comp rx=0 tx=0).stop 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f092f7fe640 1 --2- 172.21.15.111:0/2626641868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093006d6f0 0x7f093006db40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0928009d90 tx=0x7f092802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f87bd8bb33be0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7f092e7fc640 1 -- 172.21.15.111:0/2626641868 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928007740 con 0x7f093006d6f0 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7f092e7fc640 1 -- 172.21.15.111:0/2626641868 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09280078e0 con 0x7f093006d6f0 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7f092e7fc640 1 -- 172.21.15.111:0/2626641868 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928007cb0 con 0x7f093006d6f0 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a55a7d640 1 -- 172.21.15.111:0/645230573 >> v2:172.21.15.111:3300/0 conn(0x7f6a4800a520 msgr2=0x7f6a480e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a55a7d640 1 --2- 172.21.15.111:0/645230573 >> v2:172.21.15.111:3300/0 conn(0x7f6a4800a520 0x7f6a480e0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a55a7d640 1 -- 172.21.15.111:0/645230573 >> v1:172.21.15.153:6789/0 conn(0x7f6a48051720 legacy=0x7f6a48009e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7f6a55a7d640 1 -- 172.21.15.111:0/645230573 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a48052530 con 0x7f6a480e59b0 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7f6a55a7d640 1 --2- 172.21.15.111:0/645230573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a480e59b0 0x7f6a480e4d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a48003f50 tx=0x7f6a4002a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455dd9c7939357eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 -- 172.21.15.111:0/3655999005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 msgr2=0x7ff2d0162340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 --2- 172.21.15.111:0/3655999005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d0162340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009870 tx=0x7ff2c802a180 comp rx=0 tx=0).stop 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 -- 172.21.15.111:0/3655999005 shutdown_connections 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 --2- 172.21.15.111:0/3655999005 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d0162340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 --2- 172.21.15.111:0/3655999005 >> v2:172.21.15.153:3300/0 conn(0x7ff2d015e240 0x7ff2d00b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 -- 172.21.15.111:0/3655999005 >> 172.21.15.111:0/3655999005 conn(0x7ff2d00d4570 msgr2=0x7ff2d00d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 -- 172.21.15.111:0/3655999005 shutdown_connections 2024-05-27T00:39:41.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7ff2d8363640 1 -- 172.21.15.111:0/3655999005 wait complete. 2024-05-27T00:39:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 -- 172.21.15.111:0/3858348432 >> v2:172.21.15.111:3300/0 conn(0x7ff3980d0ce0 msgr2=0x7ff39806b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 --2- 172.21.15.111:0/3858348432 >> v2:172.21.15.111:3300/0 conn(0x7ff3980d0ce0 0x7ff39806b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 -- 172.21.15.111:0/3858348432 >> v1:172.21.15.111:6789/0 conn(0x7ff398068b00 legacy=0x7ff39806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.444+0000 7ff3977fe640 1 -- 172.21.15.111:0/3858348432 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39815a9d0 con 0x7ff3980af4d0 2024-05-27T00:39:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093006d6f0 msgr2=0x7f093006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.472 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 --2- 172.21.15.111:0/2626641868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093006d6f0 0x7f093006db40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0928009d90 tx=0x7f092802a660 comp rx=0 tx=0).stop 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7ff3977fe640 1 --2- 172.21.15.111:0/3858348432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff39806d6f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009870 tx=0x7ff38c02a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2250e5b15120ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f6a54a7b640 1 -- 172.21.15.111:0/645230573 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40037070 con 0x7f6a480e59b0 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fbcbd4f7640 1 --2- 172.21.15.111:0/2806718458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fbcbd4f7640 1 -- 172.21.15.111:0/2806718458 >> v1:172.21.15.153:6789/0 conn(0x7fbcb814bec0 legacy=0x7fbcb806de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fbcbd4f7640 1 -- 172.21.15.111:0/2806718458 >> v1:172.21.15.111:6789/0 conn(0x7fbcb8072190 legacy=0x7fbcb80bb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fbcbccf6640 1 -- 172.21.15.111:0/2806718458 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407646791 0 0) 0x7fbcb80bc1d0 con 0x7fbcb814bec0 2024-05-27T00:39:41.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.445+0000 7fbcbd4f7640 1 -- 172.21.15.111:0/2806718458 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb80bc4b0 con 0x7fbcb80d5d70 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f5011cd2640 1 -- 172.21.15.111:0/3685103813 shutdown_connections 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3685103813 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c0b7310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f5011cd2640 1 -- 172.21.15.111:0/3685103813 >> 172.21.15.111:0/3685103813 conn(0x7f500c0d4570 msgr2=0x7f500c0d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 shutdown_connections 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 --2- 172.21.15.111:0/2626641868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f093006d6f0 0x7f093006db40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 --2- 172.21.15.111:0/2626641868 >> v2:172.21.15.111:3300/0 conn(0x7f09300af740 0x7f093006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 >> 172.21.15.111:0/2626641868 conn(0x7f0930071b90 msgr2=0x7f09300b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f6a54a7b640 1 -- 172.21.15.111:0/645230573 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40032440 con 0x7f6a480e59b0 2024-05-27T00:39:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f5011cd2640 1 -- 172.21.15.111:0/3685103813 shutdown_connections 2024-05-27T00:39:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 shutdown_connections 2024-05-27T00:39:41.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f09357a5640 1 -- 172.21.15.111:0/2626641868 wait complete. 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 shutdown_connections 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/554809971 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc718163cb0 0x7fc71816c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/554809971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc71806b390 0x7fc7180b3740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 >> 172.21.15.111:0/554809971 conn(0x7fc7180b1c00 msgr2=0x7fc7180b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7ff395ffb640 1 -- 172.21.15.111:0/3858348432 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c037070 con 0x7ff3980af4d0 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7ff395ffb640 1 -- 172.21.15.111:0/3858348432 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c032640 con 0x7ff3980af4d0 2024-05-27T00:39:41.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7ff395ffb640 1 -- 172.21.15.111:0/3858348432 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c032a50 con 0x7ff3980af4d0 2024-05-27T00:39:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 shutdown_connections 2024-05-27T00:39:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fc71e3a5640 1 -- 172.21.15.111:0/554809971 wait complete. 2024-05-27T00:39:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7fbcbd4f7640 1 --2- 172.21.15.111:0/2806718458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d61d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097d0 tx=0x7fbca802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f88ca5abd18884f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.446+0000 7f5011cd2640 1 -- 172.21.15.111:0/3685103813 wait complete. 2024-05-27T00:39:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7fbcbccf6640 1 -- 172.21.15.111:0/2806718458 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8037070 con 0x7fbcb80d5d70 2024-05-27T00:39:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7fe11a03c640 1 Processor -- start 2024-05-27T00:39:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7fbcbccf6640 1 -- 172.21.15.111:0/2806718458 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca8032440 con 0x7fbcb80d5d70 2024-05-27T00:39:41.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7f6a56a7f640 1 -- 172.21.15.111:0/645230573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a480e59b0 msgr2=0x7f6a480e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7fe11a03c640 1 -- start start 2024-05-27T00:39:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7fbcbccf6640 1 -- 172.21.15.111:0/2806718458 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8032810 con 0x7fbcb80d5d70 2024-05-27T00:39:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff2d8363640 1 Processor -- start 2024-05-27T00:39:41.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff2d8363640 1 -- start start 2024-05-27T00:39:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff39d89e640 1 -- 172.21.15.111:0/3858348432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 msgr2=0x7ff39806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff39d89e640 1 --2- 172.21.15.111:0/3858348432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff39806d6f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009870 tx=0x7ff38c02a330 comp rx=0 tx=0).stop 2024-05-27T00:39:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff39d89e640 1 -- 172.21.15.111:0/3858348432 shutdown_connections 2024-05-27T00:39:41.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff39d89e640 1 --2- 172.21.15.111:0/3858348432 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff39806d6f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/645230573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a480e59b0 0x7f6a480e4d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a48003f50 tx=0x7f6a4002a1f0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff39d89e640 1 --2- 172.21.15.111:0/3858348432 >> v2:172.21.15.111:3300/0 conn(0x7ff3980d0ce0 0x7ff39806b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7ff39d89e640 1 -- 172.21.15.111:0/3858348432 >> 172.21.15.111:0/3858348432 conn(0x7ff398071b90 msgr2=0x7ff3980b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7f6a56a7f640 1 -- 172.21.15.111:0/645230573 shutdown_connections 2024-05-27T00:39:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/645230573 >> v2:172.21.15.111:3300/0 conn(0x7f6a4800a520 0x7f6a480e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/645230573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a480e59b0 0x7f6a480e4d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f6a56a7f640 1 -- 172.21.15.111:0/645230573 >> 172.21.15.111:0/645230573 conn(0x7f6a480537b0 msgr2=0x7f6a48053bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.447+0000 7f5011cd2640 1 Processor -- start 2024-05-27T00:39:41.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f5011cd2640 1 -- start start 2024-05-27T00:39:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f6a56a7f640 1 -- 172.21.15.111:0/645230573 shutdown_connections 2024-05-27T00:39:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f6a56a7f640 1 -- 172.21.15.111:0/645230573 wait complete. 2024-05-27T00:39:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 -- 172.21.15.111:0/2806718458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 msgr2=0x7fbcb80d61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/2806718458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d61d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097d0 tx=0x7fbca802a1c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f09357a5640 1 Processor -- start 2024-05-27T00:39:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7f09357a5640 1 -- start start 2024-05-27T00:39:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 -- 172.21.15.111:0/2806718458 shutdown_connections 2024-05-27T00:39:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/2806718458 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d61d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 -- 172.21.15.111:0/2806718458 >> 172.21.15.111:0/2806718458 conn(0x7fbcb806c290 msgr2=0x7fbcb806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fc71e3a5640 1 Processor -- start 2024-05-27T00:39:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 -- 172.21.15.111:0/2806718458 shutdown_connections 2024-05-27T00:39:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fbcbff5c640 1 -- 172.21.15.111:0/2806718458 wait complete. 2024-05-27T00:39:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fe11a03c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1140b3420 0x7fe11415c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fc71e3a5640 1 -- start start 2024-05-27T00:39:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fe11a03c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 0x7fe11414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7ff2d8363640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d006bff0 0x7ff2d00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7ff39d89e640 1 -- 172.21.15.111:0/3858348432 shutdown_connections 2024-05-27T00:39:41.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fe11a03c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11415e1b0 0x7fe114070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7ff2d8363640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 0x7ff2d00c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fe11a03c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1140d3960 con 0x7fe1140cfb50 2024-05-27T00:39:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fe11a03c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe1140d2150 con 0x7fe11415e1b0 2024-05-27T00:39:41.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7fe11a03c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe1140d3ae0 con 0x7fe1140b3420 2024-05-27T00:39:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7f5011cd2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 0x7f500c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7ff2d8363640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d00c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe11903a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1140b3420 0x7fe11415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe118839640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 0x7fe11414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7ff2d58fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 0x7ff2d00c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe118839640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 0x7fe11414f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50904/0 (socket says 172.21.15.111:50904) 2024-05-27T00:39:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe118839640 1 -- 172.21.15.111:0/2222082202 learned_addr learned my addr 172.21.15.111:0/2222082202 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe11983b640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11415e1b0 0x7fe114070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7ff2d58fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 0x7ff2d00c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56338/0 (socket says 172.21.15.111:56338) 2024-05-27T00:39:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7f5011cd2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7f09357a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f093006d6f0 0x7f09300ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7f09357a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09300af740 0x7f0930152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f09357a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 0x7f09301ef160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 0x7f500c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe118839640 1 -- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1140b3420 msgr2=0x7fe11415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.449+0000 7fe118839640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1140b3420 0x7fe11415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fe118839640 1 -- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11415e1b0 msgr2=0x7fe114070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fe118839640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11415e1b0 0x7fe114070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fe118839640 1 -- 172.21.15.111:0/2222082202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11414f950 con 0x7fe1140cfb50 2024-05-27T00:39:41.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f09357a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f09300afb20 con 0x7f093006d6f0 2024-05-27T00:39:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f09357a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f093015a9d0 con 0x7f09300d0ce0 2024-05-27T00:39:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7ff2d58fe640 1 -- 172.21.15.111:0/2461271356 learned_addr learned my addr 172.21.15.111:0/2461271356 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f09357a5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f093015b860 con 0x7f09300af740 2024-05-27T00:39:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 0x7f09301ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 0x7f09301ef160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56360/0 (socket says 172.21.15.111:56360) 2024-05-27T00:39:41.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 -- 172.21.15.111:0/4066990592 learned_addr learned my addr 172.21.15.111:0/4066990592 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fe118839640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 0x7fe11414f410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00efc0 tx=0x7fe10c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d00b7850 con 0x7ff2d006bff0 2024-05-27T00:39:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 0x7f500c15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56354/0 (socket says 172.21.15.111:56354) 2024-05-27T00:39:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f500b7fe640 1 -- 172.21.15.111:0/3601747791 learned_addr learned my addr 172.21.15.111:0/3601747791 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fc71e3a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71806b390 0x7fc71815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f500c161ec0 0x7f500c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f6a56a7f640 1 Processor -- start 2024-05-27T00:39:41.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fc71e3a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 0x7fc71814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fbcbff5c640 1 Processor -- start 2024-05-27T00:39:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f500bfff640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f500c161ec0 0x7f500c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f5003fff640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f500c0b7850 con 0x7f500c161ec0 2024-05-27T00:39:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f500c06cc20 con 0x7f500c06c700 2024-05-27T00:39:41.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f500c0b79d0 con 0x7f500c15e240 2024-05-27T00:39:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092effd640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f093006d6f0 0x7f09300ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f6a56a7f640 1 -- start start 2024-05-27T00:39:41.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fbcbff5c640 1 -- start start 2024-05-27T00:39:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.448+0000 7ff39d89e640 1 -- 172.21.15.111:0/3858348432 wait complete. 2024-05-27T00:39:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7ff39d89e640 1 Processor -- start 2024-05-27T00:39:41.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 -- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09300af740 msgr2=0x7f0930152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09300af740 0x7f0930152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 -- 172.21.15.111:0/4066990592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f093006d6f0 msgr2=0x7f09300ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f093006d6f0 0x7f09300ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092f7fe640 1 -- 172.21.15.111:0/4066990592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09300cdad0 con 0x7f09300d0ce0 2024-05-27T00:39:41.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7f092effd640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f093006d6f0 0x7f09300ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7ff2d60ff640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d006bff0 0x7ff2d00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7ff2d6900640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d00c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d006cc20 con 0x7ff2d015e240 2024-05-27T00:39:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d00b79d0 con 0x7ff2d0161ec0 2024-05-27T00:39:41.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7ff39d89e640 1 -- start start 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc71e3a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718163cb0 0x7fc7181608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc71e3a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc71806be80 con 0x7fc7180b3c80 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 0x7fc71814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 0x7fc71814ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50934/0 (socket says 172.21.15.111:50934) 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 -- 172.21.15.111:0/2445697657 learned_addr learned my addr 172.21.15.111:0/2445697657 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc718071f10 con 0x7fc718163cb0 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc71806c000 con 0x7fc71806b390 2024-05-27T00:39:41.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f500b7fe640 1 -- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 msgr2=0x7f500c15c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f500b7fe640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c15c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f500b7fe640 1 -- 172.21.15.111:0/3601747791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f500c161ec0 msgr2=0x7f500c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f500b7fe640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f500c161ec0 0x7f500c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c018020 con 0x7fe1140cfb50 2024-05-27T00:39:41.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f500b7fe640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c15cc90 con 0x7f500c06c700 2024-05-27T00:39:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.450+0000 7fc717fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71806b390 0x7fc71815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fe11a03c640 1 -- 172.21.15.111:0/2222082202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11406fe90 con 0x7fe1140cfb50 2024-05-27T00:39:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f092f7fe640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 0x7f09301ef160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0928009d90 tx=0x7f0928004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 -- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71806b390 msgr2=0x7fc71815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f500b7fe640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 0x7f500c15ce90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0097f0 tx=0x7f4ffc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f6a56a7f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a4800a520 0x7f6a480dee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fe11a03c640 1 -- 172.21.15.111:0/2222082202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe114070450 con 0x7fe1140cfb50 2024-05-27T00:39:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f6a56a7f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 0x7f6a480df970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928040070 con 0x7f09300d0ce0 2024-05-27T00:39:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928031bc0 con 0x7f09300d0ce0 2024-05-27T00:39:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928009450 con 0x7f09300d0ce0 2024-05-27T00:39:41.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09300cddb0 con 0x7f09300d0ce0 2024-05-27T00:39:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7ff2d58fe640 1 -- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 msgr2=0x7ff2d00c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7f6a56a7f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a480e59b0 0x7f6a480dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7ff2d58fe640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d00c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2d58fe640 1 -- 172.21.15.111:0/2461271356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d006bff0 msgr2=0x7ff2d00cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2d58fe640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d006bff0 0x7ff2d00cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09300ce370 con 0x7f09300d0ce0 2024-05-27T00:39:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a5527c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 0x7f6a480df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a5527c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 0x7f6a480df970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56388/0 (socket says 172.21.15.111:56388) 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2d58fe640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d015d890 con 0x7ff2d015e240 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71806b390 0x7fc71815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 -- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718163cb0 msgr2=0x7fc7181608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718163cb0 0x7fc7181608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fc7177fe640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71815f850 con 0x7fc7180b3c80 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fc7177fe640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 0x7fc71814ae30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b480 tx=0x7fc70800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.451+0000 7fbcbff5c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 0x7fbcb80b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbff5c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbff5c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb814bec0 0x7fbcb80d7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbff5c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb814c2a0 con 0x7fbcb8072190 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbff5c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb806d920 con 0x7fbcb814bec0 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbff5c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb806e550 con 0x7fbcb80d5d70 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a5527c640 1 -- 172.21.15.111:0/432294930 learned_addr learned my addr 172.21.15.111:0/432294930 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480e40f0 con 0x7f6a4800a520 2024-05-27T00:39:41.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a5627e640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a480e59b0 0x7f6a480dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2d6900640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2d60ff640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d006bff0 0x7ff2d00cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480e3f70 con 0x7f6a48051720 2024-05-27T00:39:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6a480e4270 con 0x7f6a480e59b0 2024-05-27T00:39:41.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbdcf8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 0x7fbcb80b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbdcf8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 0x7fbcb80b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50946/0 (socket says 172.21.15.111:50946) 2024-05-27T00:39:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fbcbdcf8640 1 -- 172.21.15.111:0/3127119072 learned_addr learned my addr 172.21.15.111:0/3127119072 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2d58fe640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 0x7ff2d00c8cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80090b0 tx=0x7ff2c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff39d89e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff398068b00 0x7ff39816bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc041030 con 0x7f500c06c700 2024-05-27T00:39:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff39d89e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff398204480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe10c014480 con 0x7fe1140cfb50 2024-05-27T00:39:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c014850 con 0x7fe1140cfb50 2024-05-27T00:39:41.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708014070 con 0x7fc7180b3c80 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbd4f7640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbe4f9640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb814bec0 0x7fbcb80d7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbdcf8640 1 -- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 msgr2=0x7fbcb80d73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbdcf8640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbdcf8640 1 -- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb814bec0 msgr2=0x7fbcb80d7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbdcf8640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb814bec0 0x7fbcb80d7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ffc030b30 con 0x7f500c06c700 2024-05-27T00:39:41.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc037040 con 0x7f500c06c700 2024-05-27T00:39:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8041020 con 0x7ff2d015e240 2024-05-27T00:39:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff39d89e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3980d0ce0 0x7ff3982049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fbcbdcf8640 1 -- 172.21.15.111:0/3127119072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb80b5320 con 0x7fbcb8072190 2024-05-27T00:39:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c80306c0 con 0x7ff2d015e240 2024-05-27T00:39:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8037040 con 0x7ff2d015e240 2024-05-27T00:39:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d00ce020 con 0x7ff2d015e240 2024-05-27T00:39:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c160b90 con 0x7f500c06c700 2024-05-27T00:39:41.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c15d600 con 0x7f500c06c700 2024-05-27T00:39:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7f6a55a7d640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a4800a520 0x7f6a480dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d0160310 con 0x7ff2d015e240 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.452+0000 7ff396ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff398068b00 0x7ff39816bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff3967fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff398204480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff3967fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff398204480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47888/0 (socket says 172.21.15.111:47888) 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff396ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff398068b00 0x7ff39816bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50964/0 (socket says 172.21.15.111:50964) 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff39d89e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980afa90 con 0x7ff398068b00 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff39d89e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980af910 con 0x7ff3980d0ce0 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff39d89e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff3980923e0 con 0x7ff3980af4d0 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f6a5527c640 1 -- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a480e59b0 msgr2=0x7f6a480dfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f6a5527c640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a480e59b0 0x7f6a480dfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f6a5527c640 1 -- 172.21.15.111:0/432294930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a4800a520 msgr2=0x7f6a480dee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f6a5527c640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a4800a520 0x7f6a480dee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f6a5527c640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4816b2a0 con 0x7f6a48051720 2024-05-27T00:39:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0928039040 con 0x7f09300d0ce0 2024-05-27T00:39:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.453+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc708010870 con 0x7fc7180b3c80 2024-05-27T00:39:41.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71815fb30 con 0x7fc7180b3c80 2024-05-27T00:39:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7181603d0 con 0x7fc7180b3c80 2024-05-27T00:39:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c0b5980 con 0x7f500c06c700 2024-05-27T00:39:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f6a5527c640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 0x7f6a480df970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c490 tx=0x7f6a4c00c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708014070 con 0x7fc7180b3c80 2024-05-27T00:39:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7ff3967fc640 1 -- 172.21.15.111:0/1793675598 learned_addr learned my addr 172.21.15.111:0/1793675598 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.454+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914005290 con 0x7f09300d0ce0 2024-05-27T00:39:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe10c005d10 con 0x7fe1140cfb50 2024-05-27T00:39:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3980d0ce0 0x7ff3982049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0052d0 con 0x7f6a48051720 2024-05-27T00:39:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4c005470 con 0x7f6a48051720 2024-05-27T00:39:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4816b580 con 0x7f6a48051720 2024-05-27T00:39:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4816bb40 con 0x7f6a48051720 2024-05-27T00:39:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7fc6fe7fc640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f0005290 con 0x7fc7180b3c80 2024-05-27T00:39:41.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c011ab0 con 0x7f6a48051720 2024-05-27T00:39:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 -- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 msgr2=0x7ff398204480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff398204480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 -- 172.21.15.111:0/1793675598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff398068b00 msgr2=0x7ff39816bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff398068b00 0x7ff39816bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 -- 172.21.15.111:0/1793675598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39816c2c0 con 0x7ff3980d0ce0 2024-05-27T00:39:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7ff3977fe640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3980d0ce0 0x7ff3982049c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff38c031000 tx=0x7ff38c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.456+0000 7f09277fe640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f091006bf00 0x7f091006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe114068460 con 0x7fe1140cfb50 2024-05-27T00:39:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7fe109ffb640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe0f806be60 0x7fe0f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe10c08bb40 con 0x7fe1140cfb50 2024-05-27T00:39:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.456+0000 7fe11903a640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe0f806be60 0x7fe0f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.455+0000 7fbcbdcf8640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 0x7fbcb80b74e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c420 tx=0x7fbcb400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.456+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09280ac5b0 con 0x7f09300d0ce0 2024-05-27T00:39:41.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.456+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ffc044070 con 0x7f500c06c700 2024-05-27T00:39:41.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.456+0000 7f092effd640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f091006bf00 0x7f091006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.456+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d00ce9d0 con 0x7ff2d015e240 2024-05-27T00:39:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7f092effd640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f091006bf00 0x7f091006e3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09300cd710 tx=0x7f0920008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7f5008ff9640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4fec06bf00 0x7f4fec06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2c8044020 con 0x7ff2d015e240 2024-05-27T00:39:41.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4ffc0abc30 con 0x7f500c06c700 2024-05-27T00:39:41.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7f5003fff640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4fec06bf00 0x7f4fec06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc708017070 con 0x7fc7180b3c80 2024-05-27T00:39:41.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7fe11903a640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe0f806be60 0x7fe0f806e320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe10403d000 tx=0x7fe104008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a38005290 con 0x7f6a48051720 2024-05-27T00:39:41.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.457+0000 7ff2ceffd640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff2bc06be30 0x7ff2bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7f5003fff640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4fec06bf00 0x7f4fec06e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f500c160980 tx=0x7f4ff4009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff2c80ab9a0 con 0x7ff2d015e240 2024-05-27T00:39:41.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7ff2d60ff640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff2bc06be30 0x7ff2bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7fc714ff9640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc70406c230 0x7fc70406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc708010670 con 0x7fc7180b3c80 2024-05-27T00:39:41.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7fc717fff640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc70406c230 0x7fc70406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7ff2d60ff640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff2bc06be30 0x7ff2bc06e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0008630 tx=0x7ff2c00077c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.458+0000 7fc717fff640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc70406c230 0x7fc70406e6f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc70c033000 tx=0x7fc70c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.460+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f092803a3b0 con 0x7f09300d0ce0 2024-05-27T00:39:41.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.461+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40074e0 con 0x7fbcb8072190 2024-05-27T00:39:41.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.461+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb80b35f0 con 0x7fbcb8072190 2024-05-27T00:39:41.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.461+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc708058de0 con 0x7fc7180b3c80 2024-05-27T00:39:41.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.461+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb80b3bb0 con 0x7fbcb8072190 2024-05-27T00:39:41.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.462+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb8068680 con 0x7fbcb8072190 2024-05-27T00:39:41.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.464+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f500c0b5980 con 0x7f500c06c700 2024-05-27T00:39:41.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.465+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c041020 con 0x7ff3980d0ce0 2024-05-27T00:39:41.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.465+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3980b0950 con 0x7ff3980d0ce0 2024-05-27T00:39:41.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.466+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff398204f00 con 0x7ff3980d0ce0 2024-05-27T00:39:41.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.466+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a4c011c50 con 0x7f6a48051720 2024-05-27T00:39:41.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.467+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe10c022070 con 0x7fe1140cfb50 2024-05-27T00:39:41.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.466+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb40047e0 con 0x7fbcb8072190 2024-05-27T00:39:41.534 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.467+0000 7f6a467fc640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6a3406be80 0x7f6a3406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.534 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.467+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff398099e30 con 0x7ff3980d0ce0 2024-05-27T00:39:41.534 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.467+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40010e0 con 0x7fbcb8072190 2024-05-27T00:39:41.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.468+0000 7f6a55a7d640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6a3406be80 0x7f6a3406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.469+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c032640 con 0x7ff3980d0ce0 2024-05-27T00:39:41.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.469+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c037040 con 0x7ff3980d0ce0 2024-05-27T00:39:41.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.469+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6a4c05b170 con 0x7f6a48051720 2024-05-27T00:39:41.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.469+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff2c804b030 con 0x7ff2d015e240 2024-05-27T00:39:41.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.469+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcb4011070 con 0x7fbcb8072190 2024-05-27T00:39:41.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.470+0000 7fbcae7fc640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbc9c06be60 0x7fbc9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.470+0000 7f6a55a7d640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6a3406be80 0x7f6a3406e340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a40034000 tx=0x7f6a4002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.470+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbcb408b340 con 0x7fbcb8072190 2024-05-27T00:39:41.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.471+0000 7fbcbd4f7640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbc9c06be60 0x7fbc9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.471+0000 7fbcbd4f7640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbc9c06be60 0x7fbc9c06e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbca8034000 tx=0x7fbca802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.473+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbcb40592c0 con 0x7fbcb8072190 2024-05-27T00:39:41.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.474+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6a4c05d0d0 con 0x7f6a48051720 2024-05-27T00:39:41.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.475+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff38c044020 con 0x7ff3980d0ce0 2024-05-27T00:39:41.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.476+0000 7ff37ffff640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff38406be60 0x7ff38406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:41.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.476+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff38c0abeb0 con 0x7ff3980d0ce0 2024-05-27T00:39:41.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.476+0000 7ff396ffd640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff38406be60 0x7ff38406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:41.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.477+0000 7ff396ffd640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff38406be60 0x7ff38406e320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b810 tx=0x7ff388008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:41.539 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.478+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff398099e30 con 0x7ff3980d0ce0 2024-05-27T00:39:41.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.878+0000 7fc6fe7fc640 1 -- 172.21.15.111:0/2445697657 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc6f0005850 con 0x7fc7180b3c80 2024-05-27T00:39:41.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.878+0000 7fc714ff9640 1 -- 172.21.15.111:0/2445697657 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc70805d0e0 con 0x7fc7180b3c80 2024-05-27T00:39:41.879 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc70406c230 msgr2=0x7fc70406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc70406c230 0x7fc70406e6f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc70c033000 tx=0x7fc70c008030 comp rx=0 tx=0).stop 2024-05-27T00:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 msgr2=0x7fc71814ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 0x7fc71814ae30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b480 tx=0x7fc70800b940 comp rx=0 tx=0).stop 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 shutdown_connections 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc70406c230 0x7fc70406e6f0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc70c033000 tx=0x7fc70c008030 comp rx=0 tx=0).stop 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.882+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718163cb0 0x7fc7181608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.883+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7180b3c80 0x7fc71814ae30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.883+0000 7fc71e3a5640 1 --2- 172.21.15.111:0/2445697657 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc71806b390 0x7fc71815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.883+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 >> 172.21.15.111:0/2445697657 conn(0x7fc7180b1c00 msgr2=0x7fc7181482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.883+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 shutdown_connections 2024-05-27T00:39:41.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.883+0000 7fc71e3a5640 1 -- 172.21.15.111:0/2445697657 wait complete. 2024-05-27T00:39:41.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.890+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0914005580 con 0x7f09300d0ce0 2024-05-27T00:39:41.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.891+0000 7f09277fe640 1 -- 172.21.15.111:0/4066990592 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f092807ac90 con 0x7f09300d0ce0 2024-05-27T00:39:41.892 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.892+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f500c1e92c0 con 0x7f500c06c700 2024-05-27T00:39:41.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.893+0000 7f5008ff9640 1 -- 172.21.15.111:0/3601747791 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f500c15cc90 con 0x7f500c06c700 2024-05-27T00:39:41.894 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.894+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe1140b7910 con 0x7fe1140cfb50 2024-05-27T00:39:41.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.894+0000 7fe109ffb640 1 -- 172.21.15.111:0/2222082202 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe10c010040 con 0x7fe1140cfb50 2024-05-27T00:39:41.895 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.894+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f091006bf00 msgr2=0x7f091006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.894+0000 7f09357a5640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f091006bf00 0x7f091006e3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09300cd710 tx=0x7f0920008040 comp rx=0 tx=0).stop 2024-05-27T00:39:41.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 msgr2=0x7f09301ef160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 0x7f09301ef160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0928009d90 tx=0x7f0928004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 shutdown_connections 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f091006bf00 0x7f091006e3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09300d0ce0 0x7f09301ef160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09300af740 0x7f0930152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 --2- 172.21.15.111:0/4066990592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f093006d6f0 0x7f09300ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 >> 172.21.15.111:0/4066990592 conn(0x7f0930071b90 msgr2=0x7f093015c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.894+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6a38005850 con 0x7f6a48051720 2024-05-27T00:39:41.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 shutdown_connections 2024-05-27T00:39:41.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f09357a5640 1 -- 172.21.15.111:0/4066990592 wait complete. 2024-05-27T00:39:41.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff2d00cefd0 con 0x7ff2d015e240 2024-05-27T00:39:41.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.895+0000 7f6a467fc640 1 -- 172.21.15.111:0/432294930 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a4c0613d0 con 0x7f6a48051720 2024-05-27T00:39:41.899 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.899 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2024-05-27T00:39:41.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.896+0000 7ff2ceffd640 1 -- 172.21.15.111:0/2461271356 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2c807a120 con 0x7ff2d015e240 2024-05-27T00:39:41.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.896+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff398092560 con 0x7ff3980d0ce0 2024-05-27T00:39:41.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.896+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4fec06bf00 msgr2=0x7f4fec06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.896+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4fec06bf00 0x7f4fec06e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f500c160980 tx=0x7f4ff4009060 comp rx=0 tx=0).stop 2024-05-27T00:39:41.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.896+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 msgr2=0x7f500c15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.896+0000 7ff37ffff640 1 -- 172.21.15.111:0/1793675598 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff38c04b030 con 0x7ff3980d0ce0 2024-05-27T00:39:41.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 0x7f500c15ce90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0097f0 tx=0x7f4ffc0046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.901 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 shutdown_connections 2024-05-27T00:39:41.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4fec06bf00 0x7f4fec06e3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f500c161ec0 0x7f500c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f500c15e240 0x7f500c15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 --2- 172.21.15.111:0/3601747791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f500c06c700 0x7f500c15ce90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 >> 172.21.15.111:0/3601747791 conn(0x7f500c0d4570 msgr2=0x7f500c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 shutdown_connections 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.897+0000 7f5011cd2640 1 -- 172.21.15.111:0/3601747791 wait complete. 2024-05-27T00:39:41.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.898+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe0f806be60 msgr2=0x7fe0f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.898+0000 7fe0ff7fe640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe0f806be60 0x7fe0f806e320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe10403d000 tx=0x7fe104008030 comp rx=0 tx=0).stop 2024-05-27T00:39:41.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.898+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 msgr2=0x7fe11414f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.898+0000 7fe0ff7fe640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 0x7fe11414f410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00efc0 tx=0x7fe10c00c490 comp rx=0 tx=0).stop 2024-05-27T00:39:41.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6a3406be80 msgr2=0x7f6a3406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6a3406be80 0x7f6a3406e340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a40034000 tx=0x7f6a4002e000 comp rx=0 tx=0).stop 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 shutdown_connections 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe0f806be60 0x7fe0f806e320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe11415e1b0 0x7fe114070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1140cfb50 0x7fe11414f410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 --2- 172.21.15.111:0/2222082202 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe1140b3420 0x7fe11415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 msgr2=0x7f6a480df970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 0x7f6a480df970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c490 tx=0x7f6a4c00c950 comp rx=0 tx=0).stop 2024-05-27T00:39:41.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 >> 172.21.15.111:0/2222082202 conn(0x7fe1140d4450 msgr2=0x7fe11406b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 shutdown_connections 2024-05-27T00:39:41.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7fe0ff7fe640 1 -- 172.21.15.111:0/2222082202 wait complete. 2024-05-27T00:39:41.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 shutdown_connections 2024-05-27T00:39:41.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6a3406be80 0x7f6a3406e340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6a480e59b0 0x7f6a480dfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a48051720 0x7f6a480df970 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 --2- 172.21.15.111:0/432294930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a4800a520 0x7f6a480dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 >> 172.21.15.111:0/432294930 conn(0x7f6a480537b0 msgr2=0x7f6a48035920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 shutdown_connections 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.899+0000 7f6a56a7f640 1 -- 172.21.15.111:0/432294930 wait complete. 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbcb8067560 con 0x7fbcb8072190 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff2bc06be30 msgr2=0x7ff2bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff2bc06be30 0x7ff2bc06e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0008630 tx=0x7ff2c00077c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 msgr2=0x7ff2d00c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 0x7ff2d00c8cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80090b0 tx=0x7ff2c80046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 shutdown_connections 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff2bc06be30 0x7ff2bc06e2f0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0008630 tx=0x7ff2c00077c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff2d0161ec0 0x7ff2d00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d015e240 0x7ff2d00c8cb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 --2- 172.21.15.111:0/2461271356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2d006bff0 0x7ff2d00cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 >> 172.21.15.111:0/2461271356 conn(0x7ff2d00d4570 msgr2=0x7ff2d00d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.900+0000 7fbcae7fc640 1 -- 172.21.15.111:0/3127119072 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbcb405d5c0 con 0x7fbcb8072190 2024-05-27T00:39:41.910 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:41.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff38406be60 msgr2=0x7ff38406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff38406be60 0x7ff38406e320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b810 tx=0x7ff388008040 comp rx=0 tx=0).stop 2024-05-27T00:39:41.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3980d0ce0 msgr2=0x7ff3982049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3980d0ce0 0x7ff3982049c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff38c031000 tx=0x7ff38c0046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 shutdown_connections 2024-05-27T00:39:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff38406be60 0x7ff38406e320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3980d0ce0 0x7ff3982049c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff3980af4d0 0x7ff398204480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 --2- 172.21.15.111:0/1793675598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff398068b00 0x7ff39816bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 >> 172.21.15.111:0/1793675598 conn(0x7ff398071b90 msgr2=0x7ff3980aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-05-27T00:39:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 shutdown_connections 2024-05-27T00:39:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.901+0000 7ff2d8363640 1 -- 172.21.15.111:0/2461271356 wait complete. 2024-05-27T00:39:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.902+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 shutdown_connections 2024-05-27T00:39:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.902+0000 7ff39d89e640 1 -- 172.21.15.111:0/1793675598 wait complete. 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.903+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbc9c06be60 msgr2=0x7fbc9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.903+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbc9c06be60 0x7fbc9c06e320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbca8034000 tx=0x7fbca802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.903+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 msgr2=0x7fbcb80b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.903+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 0x7fbcb80b74e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c420 tx=0x7fbcb400c8e0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 shutdown_connections 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbc9c06be60 0x7fbc9c06e320 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbca8034000 tx=0x7fbca802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb814bec0 0x7fbcb80d7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbcb80d5d70 0x7fbcb80d73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 --2- 172.21.15.111:0/3127119072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbcb8072190 0x7fbcb80b74e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:41.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 >> 172.21.15.111:0/3127119072 conn(0x7fbcb806c290 msgr2=0x7fbcb814a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:41.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.904+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 shutdown_connections 2024-05-27T00:39:41.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:41.905+0000 7fbcbff5c640 1 -- 172.21.15.111:0/3127119072 wait complete. 2024-05-27T00:39:41.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-05-27T00:39:41.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-05-27T00:39:41.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-05-27T00:39:41.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-05-27T00:39:41.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-05-27T00:39:41.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-05-27T00:39:41.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-05-27T00:39:42.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-27T00:39:42.919 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-27T00:39:42.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-27T00:39:42.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-27T00:39:42.924 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-27T00:39:42.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-27T00:39:42.927 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-27T00:39:42.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-27T00:39:43.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.078+0000 7fa147fff640 1 Processor -- start 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.078+0000 7fa147fff640 1 -- start start 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.080+0000 7fa147fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1480b1220 0x7fa1480b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.082+0000 7fa147fff640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1480b5500 con 0x7fa1480b0dd0 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.082+0000 7fa147fff640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1480b2ec0 con 0x7fa14816ac60 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.082+0000 7fa147fff640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa1480b40b0 con 0x7fa1480b1220 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.083+0000 7fa1477fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa1480b0dd0 0x7fa1480b4d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36674/0 (socket says 172.21.15.111:36674) 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.083+0000 7fa1477fe640 1 -- 172.21.15.111:0/1800032562 learned_addr learned my addr 172.21.15.111:0/1800032562 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.083+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362011094 0 0) 0x7fa1480b5500 con 0x7fa1480b0dd0 2024-05-27T00:39:43.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.083+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa130003520 con 0x7fa1480b0dd0 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2956206344 0 0) 0x7fa130003520 con 0x7fa1480b0dd0 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1480b1220 msgr2=0x7fa1480b4810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1480b1220 0x7fa1480b4810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 >> v1:172.21.15.153:6789/0 conn(0x7fa14816ac60 legacy=0x7fa1480b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1480b31a0 con 0x7fa1480b0dd0 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fa1400019f0 con 0x7fa1480b0dd0 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 >> v1:172.21.15.111:6789/0 conn(0x7fa1480b0dd0 legacy=0x7fa1480b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1300039e0 0x7fa130003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa130004800 0x7fa130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1300052e0 0x7fa13000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480b40b0 con 0x7fa1300039e0 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa13000c140 con 0x7fa130004800 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa13000c340 con 0x7fa1300052e0 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.084+0000 7fa1477fe640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1300052e0 0x7fa13000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.085+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1300039e0 0x7fa130003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.086+0000 7fa146ffd640 1 -- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1300052e0 msgr2=0x7fa13000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.086+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa1300052e0 0x7fa13000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.086+0000 7fa146ffd640 1 -- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa130004800 msgr2=0x7fa130004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.086+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa130004800 0x7fa130004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.086+0000 7fa146ffd640 1 -- 172.21.15.111:0/1800032562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa130003520 con 0x7fa1300039e0 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.087+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1300039e0 0x7fa130003de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa13c001640 tx=0x7fa130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7953e48502edb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.088+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c001240 con 0x7fa1300039e0 2024-05-27T00:39:43.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.088+0000 7fa145ffb640 1 -- 172.21.15.111:0/1800032562 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c02e070 con 0x7fa1300039e0 2024-05-27T00:39:43.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.089+0000 7fa147fff640 1 -- 172.21.15.111:0/1800032562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1300039e0 msgr2=0x7fa130003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.089+0000 7fa147fff640 1 --2- 172.21.15.111:0/1800032562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1300039e0 0x7fa130003de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa13c001640 tx=0x7fa130005830 comp rx=0 tx=0).stop 2024-05-27T00:39:43.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.089+0000 7fa1477fe640 1 -- 172.21.15.111:0/1800032562 reap_dead start 2024-05-27T00:39:43.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.089+0000 7fa147fff640 1 -- 172.21.15.111:0/1800032562 shutdown_connections 2024-05-27T00:39:43.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.089+0000 7fa147fff640 1 -- 172.21.15.111:0/1800032562 >> 172.21.15.111:0/1800032562 conn(0x7fa1480af9b0 msgr2=0x7fa1480730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.089+0000 7fa147fff640 1 -- 172.21.15.111:0/1800032562 shutdown_connections 2024-05-27T00:39:43.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.090+0000 7fa147fff640 1 -- 172.21.15.111:0/1800032562 wait complete. 2024-05-27T00:39:43.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.092+0000 7fa147fff640 1 Processor -- start 2024-05-27T00:39:43.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.092+0000 7fa147fff640 1 -- start start 2024-05-27T00:39:43.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa147fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 0x7fa1480841c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa147fff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1480aa3c0 0x7fa14815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 0x7fa1480841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 0x7fa1480841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50984/0 (socket says 172.21.15.111:50984) 2024-05-27T00:39:43.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa147fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa14815a8e0 0x7fa14816d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa147fff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa14816b040 con 0x7fa1480a9fe0 2024-05-27T00:39:43.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa147fff640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480b1680 con 0x7fa1480aa3c0 2024-05-27T00:39:43.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa147fff640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa14815ad90 con 0x7fa14815a8e0 2024-05-27T00:39:43.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.093+0000 7fa146ffd640 1 -- 172.21.15.111:0/1825249204 learned_addr learned my addr 172.21.15.111:0/1825249204 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa1477fe640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa14815a8e0 0x7fa14816d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa1467fc640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1480aa3c0 0x7fa14815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa146ffd640 1 -- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa14815a8e0 msgr2=0x7fa14816d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa14815a8e0 0x7fa14816d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa146ffd640 1 -- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1480aa3c0 msgr2=0x7fa14815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1480aa3c0 0x7fa14815a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.094+0000 7fa146ffd640 1 -- 172.21.15.111:0/1825249204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14816c600 con 0x7fa1480a9fe0 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.095+0000 7fa146ffd640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 0x7fa1480841c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007ac0 tx=0x7fa13c000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.095+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c001240 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.095+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14816c8e0 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.096+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14816cea0 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.099+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c003c10 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.099+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c007020 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.100+0000 7fa12d7fa640 1 -- 172.21.15.111:0/1825249204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148068460 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.110+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa13c044030 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.111+0000 7fa12f7fe640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa134074740 0x7fa134076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.112+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa13c0ab2e0 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.112+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa13c04b020 con 0x7fa1480a9fe0 2024-05-27T00:39:43.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.113+0000 7fa1467fc640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa134074740 0x7fa134076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.121+0000 7fa1467fc640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa134074740 0x7fa134076c00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa148073700 tx=0x7fa138009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.130+0000 7fa0d485d640 1 Processor -- start 2024-05-27T00:39:43.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.130+0000 7fa0d485d640 1 -- start start 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.131+0000 7fa0d485d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa0d0071570 0x7fa0d00b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.131+0000 7fa0d485d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0d0071ca0 0x7fa0d016b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.131+0000 7fa0d485d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 0x7fa0d016bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.131+0000 7fa0d485d640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d01727b0 con 0x7fa0d0071ca0 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.131+0000 7fa0d485d640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d00b4260 con 0x7fa0d0071570 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.131+0000 7fa0d485d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d00b4540 con 0x7fa0d016b8d0 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 0x7fa0d016bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ce59c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa0d0071570 0x7fa0d00b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 0x7fa0d016bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:47930/0 (socket says 172.21.15.111:47930) 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 -- 172.21.15.111:0/2407664083 learned_addr learned my addr 172.21.15.111:0/2407664083 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0cdd9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0d0071ca0 0x7fa0d016b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 -- 172.21.15.111:0/2407664083 >> v2:172.21.15.153:3300/0 conn(0x7fa0d0071570 msgr2=0x7fa0d00b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 --2- 172.21.15.111:0/2407664083 >> v2:172.21.15.153:3300/0 conn(0x7fa0d0071570 0x7fa0d00b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 -- 172.21.15.111:0/2407664083 >> v2:172.21.15.111:3300/0 conn(0x7fa0d0071ca0 msgr2=0x7fa0d016b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 --2- 172.21.15.111:0/2407664083 >> v2:172.21.15.111:3300/0 conn(0x7fa0d0071ca0 0x7fa0d016b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.132+0000 7fa0ced9d640 1 -- 172.21.15.111:0/2407664083 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d006b530 con 0x7fa0d016b8d0 2024-05-27T00:39:43.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.133+0000 7fa0ced9d640 1 --2- 172.21.15.111:0/2407664083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 0x7fa0d016bce0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0d006fd60 tx=0x7fa0c002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c162d21f7954da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.134+0000 7fa0cd59a640 1 -- 172.21.15.111:0/2407664083 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0037070 con 0x7fa0d016b8d0 2024-05-27T00:39:43.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.134+0000 7fa0cd59a640 1 -- 172.21.15.111:0/2407664083 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c0032440 con 0x7fa0d016b8d0 2024-05-27T00:39:43.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.134+0000 7fa0cd59a640 1 -- 172.21.15.111:0/2407664083 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0032850 con 0x7fa0d016b8d0 2024-05-27T00:39:43.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 -- 172.21.15.111:0/2407664083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 msgr2=0x7fa0d016bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2407664083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 0x7fa0d016bce0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0d006fd60 tx=0x7fa0c002a210 comp rx=0 tx=0).stop 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7f327091c640 1 Processor -- start 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7f327091c640 1 -- start start 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 -- 172.21.15.111:0/2407664083 shutdown_connections 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2407664083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d016b8d0 0x7fa0d016bce0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2407664083 >> v2:172.21.15.111:3300/0 conn(0x7fa0d0071ca0 0x7fa0d016b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2407664083 >> v2:172.21.15.153:3300/0 conn(0x7fa0d0071570 0x7fa0d00b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7fa0d485d640 1 -- 172.21.15.111:0/2407664083 >> 172.21.15.111:0/2407664083 conn(0x7fa0d00cfe50 msgr2=0x7fa0d00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7fa0d485d640 1 -- 172.21.15.111:0/2407664083 shutdown_connections 2024-05-27T00:39:43.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7f43462fc640 1 Processor -- start 2024-05-27T00:39:43.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.135+0000 7f43462fc640 1 -- start start 2024-05-27T00:39:43.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7f9951981640 1 Processor -- start 2024-05-27T00:39:43.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7f9951981640 1 -- start start 2024-05-27T00:39:43.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7f78d6ec0640 1 Processor -- start 2024-05-27T00:39:43.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7f78d6ec0640 1 -- start start 2024-05-27T00:39:43.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7f43462fc640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f43401626d0 0x7f434006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f9951981640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f99440e4500 0x7f9944057690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f327091c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f3264034bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f327091c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32640358b0 con 0x7f32640e12d0 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f327091c640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3264050d40 con 0x7f32640364c0 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f327091c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3264050b10 con 0x7f3264035310 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f326b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f32640e12d0 0x7f3264035db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36692/0 (socket says 172.21.15.111:36692) 2024-05-27T00:39:43.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f326b7fe640 1 -- 172.21.15.111:0/1728305231 learned_addr learned my addr 172.21.15.111:0/1728305231 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f78d6ec0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f78c80e7640 0x7f78c8054560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f433f7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f43400c7c30 0x7f43400d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36678/0 (socket says 172.21.15.111:36678) 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f433ffff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f43401626d0 0x7f434006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f433f7fe640 1 -- 172.21.15.111:0/2857680453 learned_addr learned my addr 172.21.15.111:0/2857680453 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f43462fc640 1 --2- 172.21.15.111:0/2857680453 >> v2:172.21.15.111:3300/0 conn(0x7f43400d3220 0x7f43400cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f434006dcc0 con 0x7f43400c7c30 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f43400c8090 con 0x7f43400d3220 2024-05-27T00:39:43.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4340162ab0 con 0x7f43401626d0 2024-05-27T00:39:43.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f78d6ec0640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f78c8054b30 0x7f78c804f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f326bfff640 1 --2- 172.21.15.111:0/1728305231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f3264034bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f326a7fc640 1 -- 172.21.15.111:0/1728305231 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414398692 0 0) 0x7f32640358b0 con 0x7f32640e12d0 2024-05-27T00:39:43.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f433ffff640 1 -- 172.21.15.111:0/2857680453 >> v2:172.21.15.111:3300/0 conn(0x7f43400d3220 msgr2=0x7f43400cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f78d6ec0640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c8050220 con 0x7f78c804fa10 2024-05-27T00:39:43.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f78d6ec0640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f78c804fe80 con 0x7f78c80e7640 2024-05-27T00:39:43.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f78d6ec0640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f78c8053970 con 0x7f78c8054b30 2024-05-27T00:39:43.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f78d66bf640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f78c804fa10 0x7f78c80eb1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47334/0 (socket says 172.21.15.111:47334) 2024-05-27T00:39:43.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f78d66bf640 1 -- 172.21.15.111:0/3774362643 learned_addr learned my addr 172.21.15.111:0/3774362643 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f326a7fc640 1 -- 172.21.15.111:0/1728305231 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f325c003520 con 0x7f32640e12d0 2024-05-27T00:39:43.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094425852 0 0) 0x7f78c8050220 con 0x7f78c804fa10 2024-05-27T00:39:43.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78bc003520 con 0x7f78c804fa10 2024-05-27T00:39:43.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f78d56bd640 1 --2- 172.21.15.111:0/3774362643 >> v2:172.21.15.153:3300/0 conn(0x7f78c8054b30 0x7f78c804f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f4344899640 1 --2- 172.21.15.111:0/2857680453 >> v2:172.21.15.111:3300/0 conn(0x7f43400d3220 0x7f43400cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f326bfff640 1 -- 172.21.15.111:0/1728305231 >> v1:172.21.15.153:6789/0 conn(0x7f32640364c0 legacy=0x7f3264051820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T00:39:43.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f326bfff640 1 -- 172.21.15.111:0/1728305231 >> v1:172.21.15.111:6789/0 conn(0x7f32640e12d0 legacy=0x7f3264035db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f326bfff640 1 -- 172.21.15.111:0/1728305231 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3264054fe0 con 0x7f3264035310 2024-05-27T00:39:43.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4117715386 0 0) 0x7f78bc003520 con 0x7f78c804fa10 2024-05-27T00:39:43.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 >> v2:172.21.15.153:3300/0 conn(0x7f78c8054b30 msgr2=0x7f78c804f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 --2- 172.21.15.111:0/3774362643 >> v2:172.21.15.153:3300/0 conn(0x7f78c8054b30 0x7f78c804f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 >> v2:172.21.15.111:3300/0 conn(0x7f78c80e7640 msgr2=0x7f78c8054560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 --2- 172.21.15.111:0/3774362643 >> v2:172.21.15.111:3300/0 conn(0x7f78c80e7640 0x7f78c8054560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c80485c0 con 0x7f78c804fa10 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f9951981640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f994403bd50 0x7f994403c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f9951981640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9944038020 con 0x7f99440e48e0 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f9951981640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f9944038300 con 0x7f994403bd50 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f9951981640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f99440072e0 con 0x7f99440e4500 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f994bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f99440e48e0 0x7f9944054940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36698/0 (socket says 172.21.15.111:36698) 2024-05-27T00:39:43.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.137+0000 7f994bfff640 1 -- 172.21.15.111:0/3793516567 learned_addr learned my addr 172.21.15.111:0/3793516567 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731030658 0 0) 0x7f9944038020 con 0x7f99440e48e0 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f993c00bd30 con 0x7f99440e48e0 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 251839190 0 0) 0x7f993c00bd30 con 0x7f99440e48e0 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 >> v2:172.21.15.153:3300/0 conn(0x7f99440e4500 msgr2=0x7f9944057690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f994b7fe640 1 --2- 172.21.15.111:0/3793516567 >> v2:172.21.15.153:3300/0 conn(0x7f99440e4500 0x7f9944057690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 >> v2:172.21.15.111:3300/0 conn(0x7f994403bd50 msgr2=0x7f994403c1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:43.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f78d00683f0 con 0x7f78c804fa10 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 >> v1:172.21.15.153:6789/0 conn(0x7f78c804fa10 legacy=0x7f78c80eb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78bc0039b0 0x7f78bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78bc004350 0x7f78bc004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f9951180640 1 --2- 172.21.15.111:0/3793516567 >> v2:172.21.15.111:3300/0 conn(0x7f994403bd50 0x7f994403c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 --2- 172.21.15.111:0/3793516567 >> v2:172.21.15.111:3300/0 conn(0x7f994403bd50 0x7f994403c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440075c0 con 0x7f99440e48e0 2024-05-27T00:39:43.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f9938001670 con 0x7f99440e48e0 2024-05-27T00:39:43.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 >> v1:172.21.15.111:6789/0 conn(0x7f99440e48e0 legacy=0x7f9944054940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78c804fe80 con 0x7f78bc0039b0 2024-05-27T00:39:43.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8053970 con 0x7f78bc004350 2024-05-27T00:39:43.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f993c00c210 0x7f993c00c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00d010 0x7f993c00d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d66bf640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f78bc00bfe0 con 0x7f78bc005020 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f993c00dbd0 0x7f993c014210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9944038300 con 0x7f993c00d010 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99440072e0 con 0x7f993c00dbd0 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f993c014b70 con 0x7f993c00c210 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.136+0000 7fa0d485d640 1 -- 172.21.15.111:0/2407664083 wait complete. 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7fa0d485d640 1 Processor -- start 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7fa0d485d640 1 -- start start 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7fa0d485d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d0071570 0x7fa0d00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7fa0d485d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d0071ca0 0x7fa0d00aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7fa0d485d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d016b8d0 0x7fa0d016e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d0071570 0x7fa0d00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d0071570 0x7fa0d00ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56478/0 (socket says 172.21.15.111:56478) 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0cdd9b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d0071ca0 0x7fa0d00aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0ce59c640 1 -- 172.21.15.111:0/2978652243 learned_addr learned my addr 172.21.15.111:0/2978652243 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0ced9d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d016b8d0 0x7fa0d016e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f78d56bd640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78bc004350 0x7f78bc004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f994bfff640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00d010 0x7f993c00d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f433ffff640 1 --2- 172.21.15.111:0/2857680453 >> v2:172.21.15.111:3300/0 conn(0x7f43400d3220 0x7f43400cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f433ffff640 1 -- 172.21.15.111:0/2857680453 >> v1:172.21.15.111:6789/0 conn(0x7f43400c7c30 legacy=0x7f43400d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7f433ffff640 1 -- 172.21.15.111:0/2857680453 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43400d1d50 con 0x7f43401626d0 2024-05-27T00:39:43.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d66bf640 1 -- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78bc004350 msgr2=0x7f78bc004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d5ebe640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78bc0039b0 0x7f78bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d66bf640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78bc004350 0x7f78bc004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d66bf640 1 -- 172.21.15.111:0/3774362643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78bc0039b0 msgr2=0x7f78bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d66bf640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78bc0039b0 0x7f78bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d66bf640 1 -- 172.21.15.111:0/3774362643 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc003520 con 0x7f78bc005020 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f995097f640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f993c00c210 0x7f993c00c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f9951180640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f993c00dbd0 0x7f993c014210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f4344899640 1 --2- 172.21.15.111:0/2857680453 >> v2:172.21.15.111:3300/0 conn(0x7f43400d3220 0x7f43400cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:43.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.139+0000 7fa0d485d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0072110 con 0x7fa0d016b8d0 2024-05-27T00:39:43.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f433ffff640 1 --2- 172.21.15.111:0/2857680453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43401626d0 0x7f434006bda0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43400b3b10 tx=0x7f433002a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f8a0cd2c27a1d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f78d5ebe640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78bc0039b0 0x7f78bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:43.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d006ba50 con 0x7fa0d0071570 2024-05-27T00:39:43.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d016e350 con 0x7fa0d0071ca0 2024-05-27T00:39:43.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f326bfff640 1 --2- 172.21.15.111:0/1728305231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f3264034bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f32580097d0 tx=0x7f325802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160675d42c2a252c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f40634fc640 1 Processor -- start 2024-05-27T00:39:43.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.138+0000 7f40634fc640 1 -- start start 2024-05-27T00:39:43.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f40634fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f40634fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f405c14c190 0x7f405c0719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f40634fc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f405c0b3cf0 con 0x7f405c14bbd0 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f40634fc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f405c0b3fd0 con 0x7f405c14c190 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f40634fc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f405c0b42b0 con 0x7f405c14b4b0 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f4061a99640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f405c14bbd0 0x7f405c166200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36708/0 (socket says 172.21.15.111:36708) 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4061a99640 1 -- 172.21.15.111:0/2653164821 learned_addr learned my addr 172.21.15.111:0/2653164821 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f326a7fc640 1 -- 172.21.15.111:0/1728305231 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258037070 con 0x7f3264035310 2024-05-27T00:39:43.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f326a7fc640 1 -- 172.21.15.111:0/1728305231 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3258032440 con 0x7f3264035310 2024-05-27T00:39:43.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f326a7fc640 1 -- 172.21.15.111:0/1728305231 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258032810 con 0x7f3264035310 2024-05-27T00:39:43.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f78d66bf640 1 -- 172.21.15.111:0/3774362643 reap_dead start 2024-05-27T00:39:43.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f995097f640 1 -- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f993c00dbd0 msgr2=0x7f993c014210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f995097f640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f993c00dbd0 0x7f993c014210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f995097f640 1 -- 172.21.15.111:0/3793516567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00d010 msgr2=0x7f993c00d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f995097f640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00d010 0x7f993c00d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f995097f640 1 -- 172.21.15.111:0/3793516567 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c00bd30 con 0x7f993c00c210 2024-05-27T00:39:43.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f433effd640 1 -- 172.21.15.111:0/2857680453 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330037070 con 0x7f43401626d0 2024-05-27T00:39:43.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f433effd640 1 -- 172.21.15.111:0/2857680453 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330032440 con 0x7f43401626d0 2024-05-27T00:39:43.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.140+0000 7f433effd640 1 -- 172.21.15.111:0/2857680453 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330032830 con 0x7f43401626d0 2024-05-27T00:39:43.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4061298640 1 --2- 172.21.15.111:0/2653164821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f9951180640 1 -- 172.21.15.111:0/3793516567 reap_dead start 2024-05-27T00:39:43.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f78d66bf640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78bc00b9c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000b60 tx=0x7f78bc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3e76fa4af1a818 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4053fff640 1 -- 172.21.15.111:0/2653164821 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181219651 0 0) 0x7f405c0b3cf0 con 0x7f405c14bbd0 2024-05-27T00:39:43.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4060a97640 1 --2- 172.21.15.111:0/2653164821 >> v2:172.21.15.111:3300/0 conn(0x7f405c14c190 0x7f405c0719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4053fff640 1 -- 172.21.15.111:0/2653164821 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4048003520 con 0x7f405c14bbd0 2024-05-27T00:39:43.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4061298640 1 -- 172.21.15.111:0/2653164821 >> v2:172.21.15.111:3300/0 conn(0x7f405c14c190 msgr2=0x7f405c0719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4061298640 1 --2- 172.21.15.111:0/2653164821 >> v2:172.21.15.111:3300/0 conn(0x7f405c14c190 0x7f405c0719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4061298640 1 -- 172.21.15.111:0/2653164821 >> v1:172.21.15.111:6789/0 conn(0x7f405c14bbd0 legacy=0x7f405c166200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43401626d0 msgr2=0x7f434006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f43462fc640 1 --2- 172.21.15.111:0/2857680453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43401626d0 0x7f434006bda0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43400b3b10 tx=0x7f433002a150 comp rx=0 tx=0).stop 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 shutdown_connections 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f43462fc640 1 --2- 172.21.15.111:0/2857680453 >> v2:172.21.15.111:3300/0 conn(0x7f43400d3220 0x7f43400cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f43462fc640 1 --2- 172.21.15.111:0/2857680453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43401626d0 0x7f434006bda0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 >> 172.21.15.111:0/2857680453 conn(0x7f43400d42c0 msgr2=0x7f43400d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0068b00 con 0x7f78bc005020 2024-05-27T00:39:43.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d00aa070 con 0x7f78bc005020 2024-05-27T00:39:43.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 shutdown_connections 2024-05-27T00:39:43.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f43462fc640 1 -- 172.21.15.111:0/2857680453 wait complete. 2024-05-27T00:39:43.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f78d4ebc640 1 -- 172.21.15.111:0/3774362643 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00a6b90 con 0x7f78bc005020 2024-05-27T00:39:43.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 -- 172.21.15.111:0/1728305231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 msgr2=0x7f3264034bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 --2- 172.21.15.111:0/1728305231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f3264034bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f32580097d0 tx=0x7f325802a1c0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0cdd9b640 1 -- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d0071570 msgr2=0x7fa0d00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 -- 172.21.15.111:0/1728305231 shutdown_connections 2024-05-27T00:39:43.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 --2- 172.21.15.111:0/1728305231 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f3264034bf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 -- 172.21.15.111:0/1728305231 >> 172.21.15.111:0/1728305231 conn(0x7f326401e6c0 msgr2=0x7f32640553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f98835b7640 1 Processor -- start 2024-05-27T00:39:43.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0cdd9b640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d0071570 0x7fa0d00ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 -- 172.21.15.111:0/1728305231 shutdown_connections 2024-05-27T00:39:43.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0cdd9b640 1 -- 172.21.15.111:0/2978652243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d016b8d0 msgr2=0x7fa0d016e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0cdd9b640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d016b8d0 0x7fa0d016e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f98835b7640 1 -- start start 2024-05-27T00:39:43.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0cdd9b640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d008b750 con 0x7fa0d0071ca0 2024-05-27T00:39:43.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0ced9d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d016b8d0 0x7fa0d016e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:43.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0cdd9b640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d0071ca0 0x7fa0d00aedc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d5f0 tx=0x7fa0c400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.141+0000 7f4061298640 1 -- 172.21.15.111:0/2653164821 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c0b4590 con 0x7f405c14b4b0 2024-05-27T00:39:43.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f78d6ec0640 1 -- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 msgr2=0x7f78bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78bc00b9c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000b60 tx=0x7f78bc005830 comp rx=0 tx=0).stop 2024-05-27T00:39:43.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f995097f640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f993c00c210 0x7f993c00c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f993c00e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2577e8a9b0b8a73a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7f327091c640 1 -- 172.21.15.111:0/1728305231 wait complete. 2024-05-27T00:39:43.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40125f0 con 0x7fa0d0071ca0 2024-05-27T00:39:43.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.142+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c4019070 con 0x7fa0d0071ca0 2024-05-27T00:39:43.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40129d0 con 0x7fa0d0071ca0 2024-05-27T00:39:43.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f4061298640 1 --2- 172.21.15.111:0/2653164821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c071470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f40580097a0 tx=0x7f405802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb94553660b388f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037070 con 0x7f993c00c210 2024-05-27T00:39:43.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d008b190 con 0x7fa0d0071ca0 2024-05-27T00:39:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994002a9f0 con 0x7f993c00c210 2024-05-27T00:39:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f994b7fe640 1 -- 172.21.15.111:0/3793516567 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99400093b0 con 0x7f993c00c210 2024-05-27T00:39:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d016d870 con 0x7fa0d0071ca0 2024-05-27T00:39:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f43462fc640 1 Processor -- start 2024-05-27T00:39:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f4053fff640 1 -- 172.21.15.111:0/2653164821 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058037070 con 0x7f405c14b4b0 2024-05-27T00:39:43.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f4053fff640 1 -- 172.21.15.111:0/2653164821 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405802ae50 con 0x7f405c14b4b0 2024-05-27T00:39:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f98835b7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c15afe0 0x7f987c0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f43462fc640 1 -- start start 2024-05-27T00:39:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.143+0000 7f98835b7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f987c0af980 0x7f987c06e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f98835b7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987c0ae7a0 con 0x7f987c15b3c0 2024-05-27T00:39:43.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f98835b7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f987c06e9a0 con 0x7f987c0af980 2024-05-27T00:39:43.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f98835b7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f987c06eeb0 con 0x7f987c15afe0 2024-05-27T00:39:43.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f4053fff640 1 -- 172.21.15.111:0/2653164821 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058032660 con 0x7f405c14b4b0 2024-05-27T00:39:43.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f78d6ec0640 1 -- 172.21.15.111:0/3774362643 shutdown_connections 2024-05-27T00:39:43.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/3774362643 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78bc00b9c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f78d6ec0640 1 -- 172.21.15.111:0/3774362643 >> 172.21.15.111:0/3774362643 conn(0x7f78c800a400 msgr2=0x7f78c800a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f9880b52640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f987c15b3c0 0x7f987c06dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36716/0 (socket says 172.21.15.111:36716) 2024-05-27T00:39:43.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f9880b52640 1 -- 172.21.15.111:0/4135827295 learned_addr learned my addr 172.21.15.111:0/4135827295 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:43.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f40634fc640 1 -- 172.21.15.111:0/2653164821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 msgr2=0x7f405c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f40634fc640 1 --2- 172.21.15.111:0/2653164821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c071470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f40580097a0 tx=0x7f405802a100 comp rx=0 tx=0).stop 2024-05-27T00:39:43.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f327091c640 1 Processor -- start 2024-05-27T00:39:43.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f9881353640 1 --2- 172.21.15.111:0/4135827295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c15afe0 0x7f987c0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.144+0000 7f327091c640 1 -- start start 2024-05-27T00:39:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9951981640 1 -- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f993c00c210 msgr2=0x7f993c00c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f43462fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43400c7c30 0x7f4340149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f43462fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43400d3220 0x7f43400cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f43462fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43401626d0 0x7f4340160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f43462fc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f434006c410 con 0x7f43401626d0 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f43462fc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f43400d2310 con 0x7f43400c7c30 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f43462fc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f43400ce430 con 0x7f43400d3220 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f78d6ec0640 1 -- 172.21.15.111:0/3774362643 shutdown_connections 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f78d6ec0640 1 -- 172.21.15.111:0/3774362643 wait complete. 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f40634fc640 1 -- 172.21.15.111:0/2653164821 shutdown_connections 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f40634fc640 1 --2- 172.21.15.111:0/2653164821 >> v2:172.21.15.111:3300/0 conn(0x7f405c14c190 0x7f405c0719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f40634fc640 1 --2- 172.21.15.111:0/2653164821 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c071470 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f40634fc640 1 -- 172.21.15.111:0/2653164821 >> 172.21.15.111:0/2653164821 conn(0x7f405c0cfe50 msgr2=0x7f405c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9881353640 1 -- 172.21.15.111:0/4135827295 >> v2:172.21.15.111:3300/0 conn(0x7f987c0af980 msgr2=0x7f987c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9881353640 1 --2- 172.21.15.111:0/4135827295 >> v2:172.21.15.111:3300/0 conn(0x7f987c0af980 0x7f987c06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9881353640 1 -- 172.21.15.111:0/4135827295 >> v1:172.21.15.111:6789/0 conn(0x7f987c15b3c0 legacy=0x7f987c06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9881353640 1 -- 172.21.15.111:0/4135827295 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c06ebd0 con 0x7f987c15afe0 2024-05-27T00:39:43.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d0068680 con 0x7fa0d0071ca0 2024-05-27T00:39:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9951981640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f993c00c210 0x7f993c00c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f993c00e040 comp rx=0 tx=0).stop 2024-05-27T00:39:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f433ffff640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43400c7c30 0x7f4340149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f4344899640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43401626d0 0x7f4340160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9951981640 1 -- 172.21.15.111:0/3793516567 shutdown_connections 2024-05-27T00:39:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9951981640 1 --2- 172.21.15.111:0/3793516567 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f993c00c210 0x7f993c00c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.145+0000 7f9951981640 1 -- 172.21.15.111:0/3793516567 >> 172.21.15.111:0/3793516567 conn(0x7f9944020040 msgr2=0x7f9944056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f327091c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f32640491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f4344899640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43401626d0 0x7f4340160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51074/0 (socket says 172.21.15.111:51074) 2024-05-27T00:39:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f4344899640 1 -- 172.21.15.111:0/2330951071 learned_addr learned my addr 172.21.15.111:0/2330951071 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f40634fc640 1 -- 172.21.15.111:0/2653164821 shutdown_connections 2024-05-27T00:39:43.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f40634fc640 1 -- 172.21.15.111:0/2653164821 wait complete. 2024-05-27T00:39:43.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f327091c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 0x7f32640d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f9951981640 1 -- 172.21.15.111:0/3793516567 shutdown_connections 2024-05-27T00:39:43.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f9951981640 1 -- 172.21.15.111:0/3793516567 wait complete. 2024-05-27T00:39:43.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f32640491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f9881353640 1 --2- 172.21.15.111:0/4135827295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c15afe0 0x7f987c0b3cb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f986c0097a0 tx=0x7f986c02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e449fa34586eab55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0c4012b70 con 0x7fa0d0071ca0 2024-05-27T00:39:43.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f32640491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48022/0 (socket says 172.21.15.111:48022) 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f326affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 0x7f32640d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f326affd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 0x7f32640d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56504/0 (socket says 172.21.15.111:56504) 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f326b7fe640 1 -- 172.21.15.111:0/1405099339 learned_addr learned my addr 172.21.15.111:0/1405099339 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32640e12d0 0x7f326404dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f9873fff640 1 -- 172.21.15.111:0/4135827295 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c037070 con 0x7f987c15afe0 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f9873fff640 1 -- 172.21.15.111:0/4135827295 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c02ae40 con 0x7f987c15afe0 2024-05-27T00:39:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f9873fff640 1 -- 172.21.15.111:0/4135827295 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c032660 con 0x7f987c15afe0 2024-05-27T00:39:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3264035130 con 0x7f32640e12d0 2024-05-27T00:39:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3264035b90 con 0x7f32640364c0 2024-05-27T00:39:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3264051300 con 0x7f3264035310 2024-05-27T00:39:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7fa0beffd640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa0b006be60 0x7fa0b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.146+0000 7f433f7fe640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43400d3220 0x7f43400cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f78d6ec0640 1 Processor -- start 2024-05-27T00:39:43.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa0c408b4e0 con 0x7fa0d0071ca0 2024-05-27T00:39:43.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f326bfff640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32640e12d0 0x7f326404dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7fa0ce59c640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa0b006be60 0x7fa0b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f98835b7640 1 -- 172.21.15.111:0/4135827295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c15afe0 msgr2=0x7f987c0b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f98835b7640 1 --2- 172.21.15.111:0/4135827295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c15afe0 0x7f987c0b3cb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f986c0097a0 tx=0x7f986c02a0f0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f326affd640 1 -- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 msgr2=0x7f32640491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f326affd640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f32640491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f326affd640 1 -- 172.21.15.111:0/1405099339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32640e12d0 msgr2=0x7f326404dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f98835b7640 1 -- 172.21.15.111:0/4135827295 shutdown_connections 2024-05-27T00:39:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f98835b7640 1 --2- 172.21.15.111:0/4135827295 >> v2:172.21.15.111:3300/0 conn(0x7f987c0af980 0x7f987c06e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f98835b7640 1 --2- 172.21.15.111:0/4135827295 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c15afe0 0x7f987c0b3cb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f98835b7640 1 -- 172.21.15.111:0/4135827295 >> 172.21.15.111:0/4135827295 conn(0x7f987c071b90 msgr2=0x7f987c0b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f78d6ec0640 1 -- start start 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433ffff640 1 -- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43400d3220 msgr2=0x7f43400cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433ffff640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43400d3220 0x7f43400cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433ffff640 1 -- 172.21.15.111:0/2330951071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43401626d0 msgr2=0x7f4340160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433ffff640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43401626d0 0x7f4340160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433ffff640 1 -- 172.21.15.111:0/2330951071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43400cd820 con 0x7f43400c7c30 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433ffff640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43400c7c30 0x7f4340149c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4330031000 tx=0x7f43300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f9951981640 1 Processor -- start 2024-05-27T00:39:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f326affd640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32640e12d0 0x7f326404dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f326affd640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32640d3e40 con 0x7f32640364c0 2024-05-27T00:39:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f326bfff640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32640e12d0 0x7f326404dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f9951981640 1 -- start start 2024-05-27T00:39:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f98835b7640 1 -- 172.21.15.111:0/4135827295 shutdown_connections 2024-05-27T00:39:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f98835b7640 1 -- 172.21.15.111:0/4135827295 wait complete. 2024-05-27T00:39:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330037050 con 0x7f43400c7c30 2024-05-27T00:39:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7fa0ce59c640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa0b006be60 0x7fa0b006e320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009a40 tx=0x7fa0c800d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4330043070 con 0x7f43400c7c30 2024-05-27T00:39:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f326affd640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 0x7f32640d3900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f326c051060 tx=0x7f326c0755c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f78d6ec0640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78c8039e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330032810 con 0x7f43400c7c30 2024-05-27T00:39:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43400cdb00 con 0x7f43400c7c30 2024-05-27T00:39:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.147+0000 7f40634fc640 1 Processor -- start 2024-05-27T00:39:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f78d6ec0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c80dc250 0x7f78c80dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f40634fc640 1 -- start start 2024-05-27T00:39:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43400ce0c0 con 0x7f43400c7c30 2024-05-27T00:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f78d6ec0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c8180710 0x7f78c8180b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.148+0000 7f78d5ebe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78c8039e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f78d5ebe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78c8039e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48036/0 (socket says 172.21.15.111:48036) 2024-05-27T00:39:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f40634fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f78d5ebe640 1 -- 172.21.15.111:0/931366945 learned_addr learned my addr 172.21.15.111:0/931366945 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8053ea0 con 0x7f78c8180710 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8054210 con 0x7f78c80dc250 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f78c80e7a20 con 0x7f78bc005020 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c0748f0 con 0x7f32640364c0 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326404e560 con 0x7f32640364c0 2024-05-27T00:39:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326404eb20 con 0x7f32640364c0 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f9951981640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00c210 0x7f9944008cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f40634fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 0x7f405c1462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f40634fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f405c14c190 0x7f405c0a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f40634fc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f405c0b3a90 con 0x7f405c14c190 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f40634fc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f405c0b3910 con 0x7f405c14bbd0 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f40634fc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f405c06c780 con 0x7f405c14b4b0 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f4061a99640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f405c14c190 0x7f405c0a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f4061298640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f9951981640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 0x7f9944009200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f4060a97640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 0x7f405c1462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c08f030 con 0x7f32640364c0 2024-05-27T00:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c074dd0 con 0x7f32640364c0 2024-05-27T00:39:43.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.149+0000 7f78d66bf640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c8180710 0x7f78c8180b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f9951981640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99441783d0 0x7f9944178860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f994bfff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 0x7f9944009200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f994bfff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 0x7f9944009200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48050/0 (socket says 172.21.15.111:48050) 2024-05-27T00:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 0x7f405c1462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56506/0 (socket says 172.21.15.111:56506) 2024-05-27T00:39:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 -- 172.21.15.111:0/303690041 learned_addr learned my addr 172.21.15.111:0/303690041 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f994bfff640 1 -- 172.21.15.111:0/744784266 learned_addr learned my addr 172.21.15.111:0/744784266 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f98835b7640 1 Processor -- start 2024-05-27T00:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9944057810 con 0x7f993c00c210 2024-05-27T00:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9944057690 con 0x7f99441783d0 2024-05-27T00:39:43.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9944057990 con 0x7f99440513a0 2024-05-27T00:39:43.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f98835b7640 1 -- start start 2024-05-27T00:39:43.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f9951180640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99441783d0 0x7f9944178860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4324005290 con 0x7f43400c7c30 2024-05-27T00:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f995097f640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00c210 0x7f9944008cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 -- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 msgr2=0x7f405c144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 -- 172.21.15.111:0/303690041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f405c14c190 msgr2=0x7f405c0a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f405c14c190 0x7f405c0a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f4060a97640 1 -- 172.21.15.111:0/303690041 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c1451c0 con 0x7f405c14bbd0 2024-05-27T00:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f78d66bf640 1 -- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 msgr2=0x7f78c8039e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f78d66bf640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78c8039e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f78d66bf640 1 -- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c80dc250 msgr2=0x7f78c80dc6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c005290 con 0x7f32640364c0 2024-05-27T00:39:43.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f995097f640 1 -- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 msgr2=0x7f9944009200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f995097f640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 0x7f9944009200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f995097f640 1 -- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99441783d0 msgr2=0x7f9944178860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f995097f640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99441783d0 0x7f9944178860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f98835b7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c0af980 0x7f987c148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f995097f640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994404a860 con 0x7f993c00c210 2024-05-27T00:39:43.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f994bfff640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 0x7f9944009200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:43.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f98835b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 0x7f987c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f98835b7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c15b3c0 0x7f987c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f98835b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f987c06b390 con 0x7f987c15afe0 2024-05-27T00:39:43.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f98835b7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0aed20 con 0x7f987c15b3c0 2024-05-27T00:39:43.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f98835b7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f987c06b510 con 0x7f987c0af980 2024-05-27T00:39:43.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f78d66bf640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c80dc250 0x7f78c80dc6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.150+0000 7f78d66bf640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c8038dc0 con 0x7f78c8180710 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f78d66bf640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c8180710 0x7f78c8180b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78d0073d80 tx=0x7f78d0069170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f326c073150 con 0x7f32640364c0 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f4060a97640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 0x7f405c1462d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f404c00d5f0 tx=0x7f404c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c0125f0 con 0x7f405c14bbd0 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c019070 con 0x7f405c14bbd0 2024-05-27T00:39:43.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c1454a0 con 0x7f405c14bbd0 2024-05-27T00:39:43.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 0x7f987c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 0x7f987c1543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51124/0 (socket says 172.21.15.111:51124) 2024-05-27T00:39:43.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 -- 172.21.15.111:0/2962423772 learned_addr learned my addr 172.21.15.111:0/2962423772 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:43.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c146fc0 con 0x7f405c14bbd0 2024-05-27T00:39:43.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0068b00 con 0x7f78c8180710 2024-05-27T00:39:43.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d004fcd0 con 0x7f78c8180710 2024-05-27T00:39:43.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00683f0 con 0x7f78c8180710 2024-05-27T00:39:43.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c80390a0 con 0x7f78c8180710 2024-05-27T00:39:43.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c0129d0 con 0x7f405c14bbd0 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.151+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4330032a70 con 0x7f43400c7c30 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f433cff9640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f432006bf00 0x7f432006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f43300abd70 con 0x7f43400c7c30 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 -- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c0af980 msgr2=0x7f987c148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c0af980 0x7f987c148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 -- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c15b3c0 msgr2=0x7f987c06c090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c15b3c0 0x7f987c06c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f9880b52640 1 -- 172.21.15.111:0/2962423772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0cd640 con 0x7f987c15afe0 2024-05-27T00:39:43.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.152+0000 7f3257fff640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f325006bf00 0x7f325006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f326c1059e0 con 0x7f32640364c0 2024-05-27T00:39:43.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f9880b52640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 0x7f987c1543a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f987800c3f0 tx=0x7f987800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa0c4059460 con 0x7fa0d0071ca0 2024-05-27T00:39:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f433f7fe640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f432006bf00 0x7f432006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c8039660 con 0x7f78c8180710 2024-05-27T00:39:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f326b7fe640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f325006bf00 0x7f325006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f995097f640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00c210 0x7f9944008cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9940031000 tx=0x7f9940004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.153+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c071f20 con 0x7f405c14bbd0 2024-05-27T00:39:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f326b7fe640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f325006bf00 0x7f325006e3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3260000f60 tx=0x7f3260001730 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878017070 con 0x7f987c15afe0 2024-05-27T00:39:43.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f433f7fe640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f432006bf00 0x7f432006e3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43400b7030 tx=0x7f4338009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9878005ce0 con 0x7f987c15afe0 2024-05-27T00:39:43.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98780072d0 con 0x7f987c15afe0 2024-05-27T00:39:43.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940042020 con 0x7f993c00c210 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9940032a50 con 0x7f993c00c210 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037040 con 0x7f993c00c210 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99440098a0 con 0x7f993c00c210 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994404adb0 con 0x7f993c00c210 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c80057e0 con 0x7f78c8180710 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.154+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c0b04c0 con 0x7f987c15afe0 2024-05-27T00:39:43.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.155+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c0b01e0 con 0x7f987c15afe0 2024-05-27T00:39:43.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.156+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944005390 con 0x7f993c00c210 2024-05-27T00:39:43.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.156+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78d00ac030 con 0x7f78c8180710 2024-05-27T00:39:43.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.156+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c068680 con 0x7f987c15afe0 2024-05-27T00:39:43.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.156+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9940045020 con 0x7f993c00c210 2024-05-27T00:39:43.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f78c67fc640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f78b406be80 0x7f78b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f78d0125890 con 0x7f78c8180710 2024-05-27T00:39:43.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f43300b0070 con 0x7f43400c7c30 2024-05-27T00:39:43.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f78d5ebe640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f78b406be80 0x7f78b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f987801a070 con 0x7f987c15afe0 2024-05-27T00:39:43.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f99497fa640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f993006be80 0x7f993006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99400ab340 con 0x7f993c00c210 2024-05-27T00:39:43.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.157+0000 7f994bfff640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f993006be80 0x7f993006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.158+0000 7f78d5ebe640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f78b406be80 0x7f78b406e340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f78cc004700 tx=0x7f78cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.158+0000 7f994bfff640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f993006be80 0x7f993006e340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99440517c0 tx=0x7f9938001e10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.158+0000 7f9871ffb640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f986006be30 0x7f986006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.158+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f987808a4d0 con 0x7f987c15afe0 2024-05-27T00:39:43.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.158+0000 7f9881353640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f986006be30 0x7f986006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.159+0000 7f9881353640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f986006be30 0x7f986006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f986c034000 tx=0x7f986c02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.161+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f32640d3e40 con 0x7f32640364c0 2024-05-27T00:39:43.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.162+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f78d00bf060 con 0x7f78c8180710 2024-05-27T00:39:43.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.163+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f404c012b70 con 0x7f405c14bbd0 2024-05-27T00:39:43.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.164+0000 7f4051ffb640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f404006be80 0x7f404006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:43.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.164+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f404c08bfd0 con 0x7f405c14bbd0 2024-05-27T00:39:43.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.164+0000 7f4061298640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f404006be80 0x7f404006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:43.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.165+0000 7f4061298640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f404006be80 0x7f404006e340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4058034000 tx=0x7f405802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:43.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.166+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f404c05a6b0 con 0x7f405c14bbd0 2024-05-27T00:39:43.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.167+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f994004c030 con 0x7f993c00c210 2024-05-27T00:39:43.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.175+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f98780583c0 con 0x7f987c15afe0 2024-05-27T00:39:43.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.519+0000 7fa12d7fa640 1 -- 172.21.15.111:0/1825249204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa1480cc000 con 0x7fa1480a9fe0 2024-05-27T00:39:43.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.520+0000 7fa12f7fe640 1 -- 172.21.15.111:0/1825249204 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa13c0799c0 con 0x7fa1480a9fe0 2024-05-27T00:39:43.521 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.523+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa134074740 msgr2=0x7fa134076c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.523+0000 7fa147fff640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa134074740 0x7fa134076c00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa148073700 tx=0x7fa138009060 comp rx=0 tx=0).stop 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.523+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 msgr2=0x7fa1480841c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.523+0000 7fa147fff640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 0x7fa1480841c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007ac0 tx=0x7fa13c000fb0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.524+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 shutdown_connections 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.524+0000 7fa147fff640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa134074740 0x7fa134076c00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.524+0000 7fa147fff640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa14815a8e0 0x7fa14816d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.524+0000 7fa147fff640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1480aa3c0 0x7fa14815a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.524+0000 7fa147fff640 1 --2- 172.21.15.111:0/1825249204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1480a9fe0 0x7fa1480841c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.524+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 >> 172.21.15.111:0/1825249204 conn(0x7fa1480af9b0 msgr2=0x7fa14806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.525+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 shutdown_connections 2024-05-27T00:39:43.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.525+0000 7fa147fff640 1 -- 172.21.15.111:0/1825249204 wait complete. 2024-05-27T00:39:43.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-05-27T00:39:43.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.556+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f324c005580 con 0x7f32640364c0 2024-05-27T00:39:43.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.556+0000 7f3257fff640 1 -- 172.21.15.111:0/1405099339 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f326c0d40c0 con 0x7f32640364c0 2024-05-27T00:39:43.557 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.558+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f325006bf00 msgr2=0x7f325006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.558+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f325006bf00 0x7f325006e3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3260000f60 tx=0x7f3260001730 comp rx=0 tx=0).stop 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.558+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 msgr2=0x7f32640d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.558+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 0x7f32640d3900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f326c051060 tx=0x7f326c0755c0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 shutdown_connections 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f325006bf00 0x7f325006e3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32640e12d0 0x7f326404dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32640364c0 0x7f32640d3900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 --2- 172.21.15.111:0/1405099339 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3264035310 0x7f32640491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 >> 172.21.15.111:0/1405099339 conn(0x7f326401e6c0 msgr2=0x7f3264032da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 shutdown_connections 2024-05-27T00:39:43.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.559+0000 7f327091c640 1 -- 172.21.15.111:0/1405099339 wait complete. 2024-05-27T00:39:43.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.566+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa0d0162270 con 0x7fa0d0071ca0 2024-05-27T00:39:43.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.566+0000 7fa0beffd640 1 -- 172.21.15.111:0/2978652243 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0c405d760 con 0x7fa0d0071ca0 2024-05-27T00:39:43.567 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa0b006be60 msgr2=0x7fa0b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa0b006be60 0x7fa0b006e320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009a40 tx=0x7fa0c800d040 comp rx=0 tx=0).stop 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d0071ca0 msgr2=0x7fa0d00aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d0071ca0 0x7fa0d00aedc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d5f0 tx=0x7fa0c400dab0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 shutdown_connections 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa0b006be60 0x7fa0b006e320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d016b8d0 0x7fa0d016e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa0d0071ca0 0x7fa0d00aedc0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d5f0 tx=0x7fa0c400dab0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 --2- 172.21.15.111:0/2978652243 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d0071570 0x7fa0d00ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 >> 172.21.15.111:0/2978652243 conn(0x7fa0d00cfe50 msgr2=0x7fa0d0161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-05-27T00:39:43.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 shutdown_connections 2024-05-27T00:39:43.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.570+0000 7fa0d485d640 1 -- 172.21.15.111:0/2978652243 wait complete. 2024-05-27T00:39:43.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.576+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4324005580 con 0x7f43400c7c30 2024-05-27T00:39:43.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.577+0000 7f433cff9640 1 -- 172.21.15.111:0/2330951071 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4330079d70 con 0x7f43400c7c30 2024-05-27T00:39:43.578 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f432006bf00 msgr2=0x7f432006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f432006bf00 0x7f432006e3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43400b7030 tx=0x7f4338009040 comp rx=0 tx=0).stop 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43400c7c30 msgr2=0x7f4340149c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43400c7c30 0x7f4340149c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4330031000 tx=0x7f43300046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 shutdown_connections 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f432006bf00 0x7f432006e3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43401626d0 0x7f4340160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f43400d3220 0x7f43400cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 --2- 172.21.15.111:0/2330951071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f43400c7c30 0x7f4340149c80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 >> 172.21.15.111:0/2330951071 conn(0x7f43400d42c0 msgr2=0x7f43400d2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 shutdown_connections 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.579+0000 7f43462fc640 1 -- 172.21.15.111:0/2330951071 wait complete. 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.580+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f78c80ea770 con 0x7f78c8180710 2024-05-27T00:39:43.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.580+0000 7f78c67fc640 1 -- 172.21.15.111:0/931366945 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f78d00f8e20 con 0x7f78c8180710 2024-05-27T00:39:43.582 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f78b406be80 msgr2=0x7f78b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f78b406be80 0x7f78b406e340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f78cc004700 tx=0x7f78cc008040 comp rx=0 tx=0).stop 2024-05-27T00:39:43.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c8180710 msgr2=0x7f78c8180b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c8180710 0x7f78c8180b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78d0073d80 tx=0x7f78d0069170 comp rx=0 tx=0).stop 2024-05-27T00:39:43.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 shutdown_connections 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f78b406be80 0x7f78b406e340 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78c8180710 0x7f78c8180b90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78c80dc250 0x7f78c80dc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 --2- 172.21.15.111:0/931366945 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f78bc005020 0x7f78c8039e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 >> 172.21.15.111:0/931366945 conn(0x7f78c800a400 msgr2=0x7f78c80e8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 shutdown_connections 2024-05-27T00:39:43.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.583+0000 7f78d6ec0640 1 -- 172.21.15.111:0/931366945 wait complete. 2024-05-27T00:39:43.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.585+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f99440104c0 con 0x7f993c00c210 2024-05-27T00:39:43.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.586+0000 7f99497fa640 1 -- 172.21.15.111:0/744784266 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9940079440 con 0x7f993c00c210 2024-05-27T00:39:43.587 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-05-27T00:39:43.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f993006be80 msgr2=0x7f993006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f993006be80 0x7f993006e340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99440517c0 tx=0x7f9938001e10 comp rx=0 tx=0).stop 2024-05-27T00:39:43.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00c210 msgr2=0x7f9944008cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00c210 0x7f9944008cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9940031000 tx=0x7f9940004700 comp rx=0 tx=0).stop 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 shutdown_connections 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f993006be80 0x7f993006e340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99441783d0 0x7f9944178860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99440513a0 0x7f9944009200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 --2- 172.21.15.111:0/744784266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f993c00c210 0x7f9944008cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 >> 172.21.15.111:0/744784266 conn(0x7f9944020040 msgr2=0x7f9944005bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 shutdown_connections 2024-05-27T00:39:43.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.588+0000 7f9951981640 1 -- 172.21.15.111:0/744784266 wait complete. 2024-05-27T00:39:43.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.589+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f405c068680 con 0x7f405c14bbd0 2024-05-27T00:39:43.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.590+0000 7f4051ffb640 1 -- 172.21.15.111:0/303690041 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f404c059fb0 con 0x7f405c14bbd0 2024-05-27T00:39:43.591 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2024-05-27T00:39:43.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f404006be80 msgr2=0x7f404006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f404006be80 0x7f404006e340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4058034000 tx=0x7f405802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:43.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 msgr2=0x7f405c1462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 0x7f405c1462d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f404c00d5f0 tx=0x7f404c00dab0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 shutdown_connections 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f404006be80 0x7f404006e340 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f405c14c190 0x7f405c0a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f405c14bbd0 0x7f405c1462d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 --2- 172.21.15.111:0/303690041 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f405c14b4b0 0x7f405c144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 >> 172.21.15.111:0/303690041 conn(0x7f405c0cfe50 msgr2=0x7f405c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 shutdown_connections 2024-05-27T00:39:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.593+0000 7f40634fc640 1 -- 172.21.15.111:0/303690041 wait complete. 2024-05-27T00:39:43.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-05-27T00:39:43.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-05-27T00:39:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.596+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f987c06ccb0 con 0x7f987c15afe0 2024-05-27T00:39:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.596+0000 7f9871ffb640 1 -- 172.21.15.111:0/2962423772 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f987805c6c0 con 0x7f987c15afe0 2024-05-27T00:39:43.597 INFO:teuthology.orchestra.run.smithi111.stdout:55834574856 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f986006be30 msgr2=0x7f986006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f986006be30 0x7f986006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f986c034000 tx=0x7f986c02c000 comp rx=0 tx=0).stop 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 msgr2=0x7f987c1543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 0x7f987c1543a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f987800c3f0 tx=0x7f987800c8b0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 shutdown_connections 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f986006be30 0x7f986006e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f987c15b3c0 0x7f987c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f987c15afe0 0x7f987c1543a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 --2- 172.21.15.111:0/2962423772 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f987c0af980 0x7f987c148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 >> 172.21.15.111:0/2962423772 conn(0x7f987c071b90 msgr2=0x7f987c06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 shutdown_connections 2024-05-27T00:39:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:43.599+0000 7f98835b7640 1 -- 172.21.15.111:0/2962423772 wait complete. 2024-05-27T00:39:43.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-05-27T00:39:43.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-05-27T00:39:43.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-05-27T00:39:44.543 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-27T00:39:44.573 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-27T00:39:44.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-27T00:39:44.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-27T00:39:44.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-27T00:39:44.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-27T00:39:44.608 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-27T00:39:44.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-27T00:39:44.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.671+0000 7f6baf379640 1 Processor -- start 2024-05-27T00:39:44.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.671+0000 7f6baf379640 1 -- start start 2024-05-27T00:39:44.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.672+0000 7f6baf379640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6ba00ead00 0x7f6ba004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.672+0000 7f6baf379640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 0x7f6ba0007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.672+0000 7f6baf379640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba0053720 con 0x7f6ba004ffd0 2024-05-27T00:39:44.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.672+0000 7f6baf379640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba0054330 con 0x7f6ba00ead00 2024-05-27T00:39:44.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.672+0000 7f6baf379640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba0053a00 con 0x7f6ba0006ea0 2024-05-27T00:39:44.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 0x7f6ba0007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 0x7f6ba0007330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48072/0 (socket says 172.21.15.111:48072) 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 -- 172.21.15.111:0/105278471 learned_addr learned my addr 172.21.15.111:0/105278471 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6bae377640 1 --2- 172.21.15.111:0/105278471 >> v2:172.21.15.153:3300/0 conn(0x7f6ba00ead00 0x7f6ba004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 -- 172.21.15.111:0/105278471 >> v2:172.21.15.153:3300/0 conn(0x7f6ba00ead00 msgr2=0x7f6ba004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 --2- 172.21.15.111:0/105278471 >> v2:172.21.15.153:3300/0 conn(0x7f6ba00ead00 0x7f6ba004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 -- 172.21.15.111:0/105278471 >> v1:172.21.15.153:6789/0 conn(0x7f6ba004ffd0 legacy=0x7f6ba0037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6baeb78640 1 -- 172.21.15.111:0/105278471 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba0053ce0 con 0x7f6ba0006ea0 2024-05-27T00:39:44.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.673+0000 7f6bae377640 1 --2- 172.21.15.111:0/105278471 >> v2:172.21.15.153:3300/0 conn(0x7f6ba00ead00 0x7f6ba004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:44.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.674+0000 7f6baeb78640 1 --2- 172.21.15.111:0/105278471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 0x7f6ba0007330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009870 tx=0x7f6b9802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccf5ae7af7957e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.675+0000 7f6bad375640 1 -- 172.21.15.111:0/105278471 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037070 con 0x7f6ba0006ea0 2024-05-27T00:39:44.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.675+0000 7f6bad375640 1 -- 172.21.15.111:0/105278471 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98032440 con 0x7f6ba0006ea0 2024-05-27T00:39:44.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.676+0000 7f6baf379640 1 -- 172.21.15.111:0/105278471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 msgr2=0x7f6ba0007330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.676+0000 7f6baf379640 1 --2- 172.21.15.111:0/105278471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 0x7f6ba0007330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009870 tx=0x7f6b9802a250 comp rx=0 tx=0).stop 2024-05-27T00:39:44.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.676+0000 7f6baf379640 1 -- 172.21.15.111:0/105278471 shutdown_connections 2024-05-27T00:39:44.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.676+0000 7f6baf379640 1 --2- 172.21.15.111:0/105278471 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba0006ea0 0x7f6ba0007330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.676+0000 7f6baf379640 1 --2- 172.21.15.111:0/105278471 >> v2:172.21.15.153:3300/0 conn(0x7f6ba00ead00 0x7f6ba004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.676+0000 7f6baf379640 1 -- 172.21.15.111:0/105278471 >> 172.21.15.111:0/105278471 conn(0x7f6ba0009fb0 msgr2=0x7f6ba000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.677+0000 7f6baf379640 1 -- 172.21.15.111:0/105278471 shutdown_connections 2024-05-27T00:39:44.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.677+0000 7f6baf379640 1 -- 172.21.15.111:0/105278471 wait complete. 2024-05-27T00:39:44.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.679+0000 7f6baf379640 1 Processor -- start 2024-05-27T00:39:44.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.679+0000 7f6baf379640 1 -- start start 2024-05-27T00:39:44.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.680+0000 7f6baf379640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba004ffd0 0x7f6ba004b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.680+0000 7f6baf379640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba00ead00 0x7f6ba00dc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6baf379640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 0x7f6ba0045970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6badb76640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba00ead00 0x7f6ba00dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6badb76640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba00ead00 0x7f6ba00dc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56550/0 (socket says 172.21.15.111:56550) 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6baeb78640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 0x7f6ba0045970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6baeb78640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 0x7f6ba0045970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51130/0 (socket says 172.21.15.111:51130) 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6badb76640 1 -- 172.21.15.111:0/477018077 learned_addr learned my addr 172.21.15.111:0/477018077 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.681+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0054940 con 0x7f6ba0049850 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba00547c0 con 0x7f6ba00ead00 2024-05-27T00:39:44.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba004f550 con 0x7f6ba004ffd0 2024-05-27T00:39:44.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baeb78640 1 -- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba004ffd0 msgr2=0x7f6ba004b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:39:44.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baeb78640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba004ffd0 0x7f6ba004b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baeb78640 1 -- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba00ead00 msgr2=0x7f6ba00dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baeb78640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba00ead00 0x7f6ba00dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.682+0000 7f6baeb78640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba0044f00 con 0x7f6ba0049850 2024-05-27T00:39:44.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.683+0000 7f6baeb78640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 0x7f6ba0045970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b98031000 tx=0x7f6b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.684+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98041020 con 0x7f6ba0049850 2024-05-27T00:39:44.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.684+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba00451e0 con 0x7f6ba0049850 2024-05-27T00:39:44.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.684+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba0049050 con 0x7f6ba0049850 2024-05-27T00:39:44.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.687+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98032440 con 0x7f6ba0049850 2024-05-27T00:39:44.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.687+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037040 con 0x7f6ba0049850 2024-05-27T00:39:44.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.688+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b98044020 con 0x7f6ba0049850 2024-05-27T00:39:44.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.689+0000 7f6b9effd640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6b8c06bf30 0x7f6b8c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.690+0000 7f6bae377640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6b8c06bf30 0x7f6b8c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.690+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6b980abb20 con 0x7f6ba0049850 2024-05-27T00:39:44.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.692+0000 7f6bae377640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6b8c06bf30 0x7f6b8c06e3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba804fc80 tx=0x7f6ba806b2c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.692+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b90005290 con 0x7f6ba0049850 2024-05-27T00:39:44.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.701+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6b9804b030 con 0x7f6ba0049850 2024-05-27T00:39:44.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.734+0000 7f46a9a43640 1 Processor -- start 2024-05-27T00:39:44.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.734+0000 7f46a9a43640 1 -- start start 2024-05-27T00:39:44.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.735+0000 7f46a9a43640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f469c050120 0x7f469c054510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.735+0000 7f46a9a43640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469c007770 con 0x7f469c054d80 2024-05-27T00:39:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.735+0000 7f46a9a43640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469c007a50 con 0x7f469c0387f0 2024-05-27T00:39:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.735+0000 7f46a9a43640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f469c007520 con 0x7f469c050120 2024-05-27T00:39:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f46a8a41640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f469c050120 0x7f469c054510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f46a8a41640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f469c050120 0x7f469c054510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51134/0 (socket says 172.21.15.111:51134) 2024-05-27T00:39:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f46a8a41640 1 -- 172.21.15.111:0/1315157766 learned_addr learned my addr 172.21.15.111:0/1315157766 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.735+0000 7f2c7d626640 1 Processor -- start 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.735+0000 7f2c7d626640 1 -- start start 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f2c7d626640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2c7814b370 0x7f2c78072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f2c7d626640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c780cd4f0 con 0x7f2c7814c020 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f2c7d626640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7814b750 con 0x7f2c780b3750 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.736+0000 7f2c7d626640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2c780b3be0 con 0x7f2c7814b370 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.737+0000 7f2c777fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2c7814c020 0x7f2c780b4be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36722/0 (socket says 172.21.15.111:36722) 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.737+0000 7f2c777fe640 1 -- 172.21.15.111:0/224588032 learned_addr learned my addr 172.21.15.111:0/224588032 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.737+0000 7f2c77fff640 1 --2- 172.21.15.111:0/224588032 >> v2:172.21.15.153:3300/0 conn(0x7f2c7814b370 0x7f2c78072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.737+0000 7f46a8a41640 1 -- 172.21.15.111:0/1315157766 >> v1:172.21.15.153:6789/0 conn(0x7f469c0387f0 legacy=0x7f469c038c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:44.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.737+0000 7f46a8a41640 1 -- 172.21.15.111:0/1315157766 >> v1:172.21.15.111:6789/0 conn(0x7f469c054d80 legacy=0x7f469c050500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:44.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.737+0000 7f46a8a41640 1 -- 172.21.15.111:0/1315157766 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c007d30 con 0x7f469c050120 2024-05-27T00:39:44.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.738+0000 7f2c77fff640 1 -- 172.21.15.111:0/224588032 >> v1:172.21.15.153:6789/0 conn(0x7f2c780b3750 legacy=0x7f2c78071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.738+0000 7f2c77fff640 1 -- 172.21.15.111:0/224588032 >> v1:172.21.15.111:6789/0 conn(0x7f2c7814c020 legacy=0x7f2c780b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.738+0000 7f2c77fff640 1 -- 172.21.15.111:0/224588032 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c780b3ec0 con 0x7f2c7814b370 2024-05-27T00:39:44.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.738+0000 7f46a8a41640 1 --2- 172.21.15.111:0/1315157766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c050120 0x7f469c054510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46a4065130 tx=0x7f46a40a6920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f76651e8e3571d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.738+0000 7f46a37fe640 1 -- 172.21.15.111:0/1315157766 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a40a9070 con 0x7f469c050120 2024-05-27T00:39:44.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.739+0000 7f46a37fe640 1 -- 172.21.15.111:0/1315157766 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a40ad470 con 0x7f469c050120 2024-05-27T00:39:44.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.739+0000 7f46a37fe640 1 -- 172.21.15.111:0/1315157766 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a40ad8e0 con 0x7f469c050120 2024-05-27T00:39:44.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.739+0000 7f2c77fff640 1 --2- 172.21.15.111:0/224588032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7814b370 0x7f2c78072fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0097a0 tx=0x7f2c6c029fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9645c3072334978 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.739+0000 7f46a9a43640 1 -- 172.21.15.111:0/1315157766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c050120 msgr2=0x7f469c054510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.739+0000 7f46a9a43640 1 --2- 172.21.15.111:0/1315157766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c050120 0x7f469c054510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46a4065130 tx=0x7f46a40a6920 comp rx=0 tx=0).stop 2024-05-27T00:39:44.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f46a9a43640 1 -- 172.21.15.111:0/1315157766 shutdown_connections 2024-05-27T00:39:44.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f46a9a43640 1 --2- 172.21.15.111:0/1315157766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c050120 0x7f469c054510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f46a9a43640 1 -- 172.21.15.111:0/1315157766 >> 172.21.15.111:0/1315157766 conn(0x7f469c055350 msgr2=0x7f469c055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f46a9a43640 1 -- 172.21.15.111:0/1315157766 shutdown_connections 2024-05-27T00:39:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f2c76ffd640 1 -- 172.21.15.111:0/224588032 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c037070 con 0x7f2c7814b370 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f2c76ffd640 1 -- 172.21.15.111:0/224588032 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c6c02ac40 con 0x7f2c7814b370 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f2c76ffd640 1 -- 172.21.15.111:0/224588032 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c032440 con 0x7f2c7814b370 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 -- 172.21.15.111:0/224588032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7814b370 msgr2=0x7f2c78072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 --2- 172.21.15.111:0/224588032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7814b370 0x7f2c78072fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0097a0 tx=0x7f2c6c029fb0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 -- 172.21.15.111:0/224588032 shutdown_connections 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 --2- 172.21.15.111:0/224588032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c7814b370 0x7f2c78072fa0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0097a0 tx=0x7f2c6c029fb0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 -- 172.21.15.111:0/224588032 >> 172.21.15.111:0/224588032 conn(0x7f2c7806d300 msgr2=0x7f2c7806d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.740+0000 7f46a9a43640 1 -- 172.21.15.111:0/1315157766 wait complete. 2024-05-27T00:39:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 -- 172.21.15.111:0/224588032 shutdown_connections 2024-05-27T00:39:44.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.741+0000 7f2c7d626640 1 -- 172.21.15.111:0/224588032 wait complete. 2024-05-27T00:39:44.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.743+0000 7f2c7d626640 1 Processor -- start 2024-05-27T00:39:44.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.743+0000 7f2c7d626640 1 -- start start 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.744+0000 7f2c7d626640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7814c020 0x7f2c78162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.744+0000 7f2c7d626640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 0x7f2c78152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c7d626640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c781532a0 0x7f2c78087ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c7d626640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78071410 con 0x7f2c78134730 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c7d626640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78071130 con 0x7f2c781532a0 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c7d626640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2c780b52f0 con 0x7f2c7814c020 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 0x7f2c78152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c777fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 0x7f2c78152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51158/0 (socket says 172.21.15.111:51158) 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f2c777fe640 1 -- 172.21.15.111:0/3703239377 learned_addr learned my addr 172.21.15.111:0/3703239377 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f2c777fe640 1 -- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7814c020 msgr2=0x7f2c78162340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T00:39:44.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f2c777fe640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7814c020 0x7f2c78162340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f2c777fe640 1 -- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c781532a0 msgr2=0x7f2c78087ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T00:39:44.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f2c777fe640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c781532a0 0x7f2c78087ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f2c777fe640 1 -- 172.21.15.111:0/3703239377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7808b980 con 0x7f2c78134730 2024-05-27T00:39:44.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.743+0000 7f46a9a43640 1 Processor -- start 2024-05-27T00:39:44.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f2c777fe640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 0x7f2c78152d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800cac0 tx=0x7f2c6800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.743+0000 7f46a9a43640 1 -- start start 2024-05-27T00:39:44.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.747+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68018070 con 0x7f2c78134730 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.744+0000 7f46a9a43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 0x7f469c185ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.744+0000 7f46a8a41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 0x7f469c185ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.744+0000 7f46a9a43640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c050120 0x7f469c035750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.744+0000 7f46a9a43640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c054d80 0x7f469c006670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f46a9a43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f469c0dfa10 con 0x7f469c0387f0 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f46a9a43640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f469c0df890 con 0x7f469c050120 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f46a9a43640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f469c006280 con 0x7f469c054d80 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f46a8a41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 0x7f469c185ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51144/0 (socket says 172.21.15.111:51144) 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.745+0000 7f46a8a41640 1 -- 172.21.15.111:0/534539070 learned_addr learned my addr 172.21.15.111:0/534539070 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.746+0000 7f46a9242640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c054d80 0x7f469c006670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.748+0000 7f46a9242640 1 -- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c050120 msgr2=0x7f469c035750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.748+0000 7f46a9242640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c050120 0x7f469c035750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.748+0000 7f46a9242640 1 -- 172.21.15.111:0/534539070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 msgr2=0x7f469c185ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f46a9242640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 0x7f469c185ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f46a9242640 1 -- 172.21.15.111:0/534539070 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c186020 con 0x7f469c054d80 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f46a8a41640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 0x7f469c185ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f46a9242640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c054d80 0x7f469c006670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f469800b3e0 tx=0x7f469800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.750+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698015020 con 0x7f469c054d80 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.750+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c035fa0 con 0x7f469c054d80 2024-05-27T00:39:44.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.750+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c036560 con 0x7f469c054d80 2024-05-27T00:39:44.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.752+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4698011830 con 0x7f469c054d80 2024-05-27T00:39:44.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.752+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c009280 con 0x7f469c054d80 2024-05-27T00:39:44.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.752+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698011ca0 con 0x7f469c054d80 2024-05-27T00:39:44.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.747+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c78164e50 con 0x7f2c78134730 2024-05-27T00:39:44.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.747+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c78165410 con 0x7f2c78134730 2024-05-27T00:39:44.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c78068460 con 0x7f2c78134730 2024-05-27T00:39:44.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c680072d0 con 0x7f2c78134730 2024-05-27T00:39:44.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.749+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68007680 con 0x7f2c78134730 2024-05-27T00:39:44.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.755+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4698018020 con 0x7f469c054d80 2024-05-27T00:39:44.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.755+0000 7f46a17fa640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f468806be30 0x7f468806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.755+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f469808b5e0 con 0x7f469c054d80 2024-05-27T00:39:44.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.756+0000 7f46a8a41640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f468806be30 0x7f468806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.756+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c6801b070 con 0x7f2c78134730 2024-05-27T00:39:44.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.757+0000 7f2c74ff9640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c6406c0d0 0x7f2c6406e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.757+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c6808a620 con 0x7f2c78134730 2024-05-27T00:39:44.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.758+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f469801f020 con 0x7f469c054d80 2024-05-27T00:39:44.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.758+0000 7f2c77fff640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c6406c0d0 0x7f2c6406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.760+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2c780cd4f0 con 0x7f2c78134730 2024-05-27T00:39:44.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.775+0000 7f46a8a41640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f468806be30 0x7f468806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46a404fc80 tx=0x7f46a40ab000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.786+0000 7f2c77fff640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c6406c0d0 0x7f2c6406e590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c034000 tx=0x7f2c6c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.801+0000 7f02d0de9640 1 Processor -- start 2024-05-27T00:39:44.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.801+0000 7f02d0de9640 1 -- start start 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02d0de9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f02c400dc60 0x7f02c40e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02d0de9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c40e3610 0x7f02c400e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02d0de9640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c40e7330 con 0x7f02c4035cc0 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02d0de9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f02c400e580 con 0x7f02c400dc60 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02d0de9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f02c40e75d0 con 0x7f02c40e3610 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02ca59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f02c400dc60 0x7f02c40e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02ca59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f02c400dc60 0x7f02c40e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51174/0 (socket says 172.21.15.111:51174) 2024-05-27T00:39:44.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.803+0000 7f02ca59c640 1 -- 172.21.15.111:0/2185259501 learned_addr learned my addr 172.21.15.111:0/2185259501 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.802+0000 7f291af47640 1 Processor -- start 2024-05-27T00:39:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.802+0000 7f291af47640 1 -- start start 2024-05-27T00:39:44.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291af47640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f290c033920 0x7f290c037c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291af47640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c054430 0x7f290c04f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291af47640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f290c052cf0 con 0x7f290c100f80 2024-05-27T00:39:44.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f02ca59c640 1 -- 172.21.15.111:0/2185259501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c40e3610 msgr2=0x7f02c400e040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:44.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f02ca59c640 1 --2- 172.21.15.111:0/2185259501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c40e3610 0x7f02c400e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f02ca59c640 1 -- 172.21.15.111:0/2185259501 >> v1:172.21.15.153:6789/0 conn(0x7f02c4035cc0 legacy=0x7f02c40e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:44.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f02ca59c640 1 -- 172.21.15.111:0/2185259501 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c40e78b0 con 0x7f02c400dc60 2024-05-27T00:39:44.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291af47640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f290c0341c0 con 0x7f290c033920 2024-05-27T00:39:44.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291af47640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f290c0548a0 con 0x7f290c054430 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291a746640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c054430 0x7f290c04f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291a746640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c054430 0x7f290c04f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48102/0 (socket says 172.21.15.111:48102) 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.804+0000 7f291a746640 1 -- 172.21.15.111:0/205275463 learned_addr learned my addr 172.21.15.111:0/205275463 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f2919f45640 1 --2- 172.21.15.111:0/205275463 >> v2:172.21.15.111:3300/0 conn(0x7f290c033920 0x7f290c037c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f2919f45640 1 -- 172.21.15.111:0/205275463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c054430 msgr2=0x7f290c04f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f2919f45640 1 --2- 172.21.15.111:0/205275463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c054430 0x7f290c04f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f2919f45640 1 -- 172.21.15.111:0/205275463 >> v1:172.21.15.153:6789/0 conn(0x7f290c100f80 legacy=0x7f290c053d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f2919f45640 1 -- 172.21.15.111:0/205275463 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c052f60 con 0x7f290c033920 2024-05-27T00:39:44.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f4a48899640 1 Processor -- start 2024-05-27T00:39:44.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.805+0000 7f4a48899640 1 -- start start 2024-05-27T00:39:44.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.806+0000 7f02ca59c640 1 --2- 172.21.15.111:0/2185259501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c400dc60 0x7f02c40e6df0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009870 tx=0x7f02bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba31d4b9d24577f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.806+0000 7f2919f45640 1 --2- 172.21.15.111:0/205275463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c033920 0x7f290c037c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2914064ec0 tx=0x7f2914094650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bd623946022b20 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.806+0000 7f02c959a640 1 -- 172.21.15.111:0/2185259501 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc037070 con 0x7f02c400dc60 2024-05-27T00:39:44.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.806+0000 7f02c959a640 1 -- 172.21.15.111:0/2185259501 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02bc032440 con 0x7f02c400dc60 2024-05-27T00:39:44.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.806+0000 7f02c959a640 1 -- 172.21.15.111:0/2185259501 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc032810 con 0x7f02c400dc60 2024-05-27T00:39:44.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a48899640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a3c0eabc0 0x7f4a3c04f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a48899640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c054670 0x7f4a3c04fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a48899640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3c034440 con 0x7f4a3c03a380 2024-05-27T00:39:44.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f02d0de9640 1 -- 172.21.15.111:0/2185259501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c400dc60 msgr2=0x7f02c40e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2185259501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c400dc60 0x7f02c40e6df0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009870 tx=0x7f02bc02a230 comp rx=0 tx=0).stop 2024-05-27T00:39:44.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f11d1e53640 1 Processor -- start 2024-05-27T00:39:44.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f11d1e53640 1 -- start start 2024-05-27T00:39:44.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f131d349640 1 Processor -- start 2024-05-27T00:39:44.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f131d349640 1 -- start start 2024-05-27T00:39:44.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f2918f43640 1 -- 172.21.15.111:0/205275463 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140a0070 con 0x7f290c033920 2024-05-27T00:39:44.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f2918f43640 1 -- 172.21.15.111:0/205275463 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914099810 con 0x7f290c033920 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a48899640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3c03b000 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a48899640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c034720 con 0x7f4a3c054670 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a4259c640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4a3c03a380 0x7f4a3c0064c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47400/0 (socket says 172.21.15.111:47400) 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a42d9d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a3c0eabc0 0x7f4a3c04f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a4259c640 1 -- 172.21.15.111:0/1934762712 learned_addr learned my addr 172.21.15.111:0/1934762712 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.807+0000 7f4a4909a640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c054670 0x7f4a3c04fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f4a41d9b640 1 -- 172.21.15.111:0/1934762712 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903679414 0 0) 0x7f4a3c034440 con 0x7f4a3c03a380 2024-05-27T00:39:44.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f4a42d9d640 1 -- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c054670 msgr2=0x7f4a3c04fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:44.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f4a42d9d640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c054670 0x7f4a3c04fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f4a42d9d640 1 -- 172.21.15.111:0/1934762712 >> v1:172.21.15.153:6789/0 conn(0x7f4a3c03a380 legacy=0x7f4a3c0064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f4a42d9d640 1 -- 172.21.15.111:0/1934762712 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c052e60 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f4a4909a640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c054670 0x7f4a3c04fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f291af47640 1 -- 172.21.15.111:0/205275463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c033920 msgr2=0x7f290c037c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f291af47640 1 --2- 172.21.15.111:0/205275463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c033920 0x7f290c037c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2914064ec0 tx=0x7f2914094650 comp rx=0 tx=0).stop 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f291af47640 1 -- 172.21.15.111:0/205275463 shutdown_connections 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f291af47640 1 --2- 172.21.15.111:0/205275463 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c054430 0x7f290c04f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f291af47640 1 --2- 172.21.15.111:0/205275463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c033920 0x7f290c037c40 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2914064ec0 tx=0x7f2914094650 comp rx=0 tx=0).stop 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.808+0000 7f291af47640 1 -- 172.21.15.111:0/205275463 >> 172.21.15.111:0/205275463 conn(0x7f290c009fb0 msgr2=0x7f290c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f291af47640 1 -- 172.21.15.111:0/205275463 shutdown_connections 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f291af47640 1 -- 172.21.15.111:0/205275463 wait complete. 2024-05-27T00:39:44.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11d1e53640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f11cc0d2c20 0x7f11cc0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11d1e53640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc0cec70 con 0x7f11cc166820 2024-05-27T00:39:44.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11d1e53640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc0cef50 con 0x7f11cc06d510 2024-05-27T00:39:44.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11d1e53640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0cf510 con 0x7f11cc0d2c20 2024-05-27T00:39:44.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131d349640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f13100dfef0 0x7f13100085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131d349640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f13100e02d0 0x7f13100e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f02d0de9640 1 -- 172.21.15.111:0/2185259501 shutdown_connections 2024-05-27T00:39:44.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2185259501 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c40e3610 0x7f02c400e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2185259501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c400dc60 0x7f02c40e6df0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f02d0de9640 1 -- 172.21.15.111:0/2185259501 >> 172.21.15.111:0/2185259501 conn(0x7f02c4056ce0 msgr2=0x7f02c40570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131d349640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13100e4510 con 0x7f13100e4100 2024-05-27T00:39:44.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131d349640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f13100094b0 con 0x7f13100e02d0 2024-05-27T00:39:44.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131d349640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1310056590 con 0x7f13100dfef0 2024-05-27T00:39:44.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f4a42d9d640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c0eabc0 0x7f4a3c04f740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097a0 tx=0x7f4a3802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b27d26ffa32e45a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11caffd640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f11cc06d510 0x7f11cc0d3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47408/0 (socket says 172.21.15.111:47408) 2024-05-27T00:39:44.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11caffd640 1 -- 172.21.15.111:0/2583964021 learned_addr learned my addr 172.21.15.111:0/2583964021 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:44.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f11cb7fe640 1 --2- 172.21.15.111:0/2583964021 >> v2:172.21.15.153:3300/0 conn(0x7f11cc0d2c20 0x7f11cc0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131cb48640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f13100e4100 0x7f131003a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47422/0 (socket says 172.21.15.111:47422) 2024-05-27T00:39:44.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f131cb48640 1 -- 172.21.15.111:0/991528140 learned_addr learned my addr 172.21.15.111:0/991528140 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:44.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.809+0000 7f4a41d9b640 1 -- 172.21.15.111:0/1934762712 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38037070 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f1317fff640 1 --2- 172.21.15.111:0/991528140 >> v2:172.21.15.153:3300/0 conn(0x7f13100dfef0 0x7f13100085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f11cb7fe640 1 -- 172.21.15.111:0/2583964021 >> v1:172.21.15.153:6789/0 conn(0x7f11cc06d510 legacy=0x7f11cc0d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f11cb7fe640 1 -- 172.21.15.111:0/2583964021 >> v1:172.21.15.111:6789/0 conn(0x7f11cc166820 legacy=0x7f11cc162900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T00:39:44.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f4a41d9b640 1 -- 172.21.15.111:0/1934762712 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38032440 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f4a41d9b640 1 -- 172.21.15.111:0/1934762712 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38032810 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f13177fe640 1 --2- 172.21.15.111:0/991528140 >> v2:172.21.15.111:3300/0 conn(0x7f13100e02d0 0x7f13100e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f1317fff640 1 -- 172.21.15.111:0/991528140 >> v2:172.21.15.111:3300/0 conn(0x7f13100e02d0 msgr2=0x7f13100e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:44.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f1317fff640 1 --2- 172.21.15.111:0/991528140 >> v2:172.21.15.111:3300/0 conn(0x7f13100e02d0 0x7f13100e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f1317fff640 1 -- 172.21.15.111:0/991528140 >> v1:172.21.15.153:6789/0 conn(0x7f13100e4100 legacy=0x7f131003a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f1316ffd640 1 -- 172.21.15.111:0/991528140 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306982806 0 0) 0x7f1318069560 con 0x7f13100e4100 2024-05-27T00:39:44.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f291af47640 1 Processor -- start 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 -- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c0eabc0 msgr2=0x7f4a3c04f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c0eabc0 0x7f4a3c04f740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097a0 tx=0x7f4a3802a180 comp rx=0 tx=0).stop 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 -- 172.21.15.111:0/1934762712 shutdown_connections 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c054670 0x7f4a3c04fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 --2- 172.21.15.111:0/1934762712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c0eabc0 0x7f4a3c04f740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 -- 172.21.15.111:0/1934762712 >> 172.21.15.111:0/1934762712 conn(0x7f4a3c0550f0 msgr2=0x7f4a3c055500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 -- 172.21.15.111:0/1934762712 shutdown_connections 2024-05-27T00:39:44.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f4a48899640 1 -- 172.21.15.111:0/1934762712 wait complete. 2024-05-27T00:39:44.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f1317fff640 1 -- 172.21.15.111:0/991528140 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1310056870 con 0x7f13100dfef0 2024-05-27T00:39:44.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.810+0000 7f11cb7fe640 1 -- 172.21.15.111:0/2583964021 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0cf230 con 0x7f11cc0d2c20 2024-05-27T00:39:44.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f1317fff640 1 --2- 172.21.15.111:0/991528140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100dfef0 0x7f13100085d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1310038840 tx=0x7f130402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca7fd4c5f2f1550 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f02d0de9640 1 -- 172.21.15.111:0/2185259501 shutdown_connections 2024-05-27T00:39:44.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f11cb7fe640 1 --2- 172.21.15.111:0/2583964021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc0d2c20 0x7f11cc0ce690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009d90 tx=0x7f11c402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e8510c04a45ab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f1316ffd640 1 -- 172.21.15.111:0/991528140 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304037070 con 0x7f13100dfef0 2024-05-27T00:39:44.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f1316ffd640 1 -- 172.21.15.111:0/991528140 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1304032440 con 0x7f13100dfef0 2024-05-27T00:39:44.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f1316ffd640 1 -- 172.21.15.111:0/991528140 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304032810 con 0x7f13100dfef0 2024-05-27T00:39:44.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f11ca7fc640 1 -- 172.21.15.111:0/2583964021 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007740 con 0x7f11cc0d2c20 2024-05-27T00:39:44.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.811+0000 7f291af47640 1 -- start start 2024-05-27T00:39:44.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291af47640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c054430 0x7f290c0f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291af47640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f290c100f80 0x7f290c0f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291af47640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 0x7f290c0f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f11ca7fc640 1 -- 172.21.15.111:0/2583964021 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c40078e0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f11ca7fc640 1 -- 172.21.15.111:0/2583964021 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007cb0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291af47640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f290c053800 con 0x7f290c054430 2024-05-27T00:39:44.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291af47640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f290c053520 con 0x7f290c100f80 2024-05-27T00:39:44.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291af47640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f290c038180 con 0x7f290c19f3e0 2024-05-27T00:39:44.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f2919744640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f290c100f80 0x7f290c0f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291a746640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 0x7f290c0f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291a746640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 0x7f290c0f9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48116/0 (socket says 172.21.15.111:48116) 2024-05-27T00:39:44.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f291a746640 1 -- 172.21.15.111:0/869816511 learned_addr learned my addr 172.21.15.111:0/869816511 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 -- 172.21.15.111:0/991528140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100dfef0 msgr2=0x7f13100085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 --2- 172.21.15.111:0/991528140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100dfef0 0x7f13100085d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1310038840 tx=0x7f130402a270 comp rx=0 tx=0).stop 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 -- 172.21.15.111:0/991528140 shutdown_connections 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 --2- 172.21.15.111:0/991528140 >> v2:172.21.15.111:3300/0 conn(0x7f13100e02d0 0x7f13100e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 --2- 172.21.15.111:0/991528140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100dfef0 0x7f13100085d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 -- 172.21.15.111:0/991528140 >> 172.21.15.111:0/991528140 conn(0x7f131001ef40 msgr2=0x7f1310055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 -- 172.21.15.111:0/991528140 shutdown_connections 2024-05-27T00:39:44.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 -- 172.21.15.111:0/2583964021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc0d2c20 msgr2=0x7f11cc0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2583964021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc0d2c20 0x7f11cc0ce690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009d90 tx=0x7f11c402a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 -- 172.21.15.111:0/2583964021 shutdown_connections 2024-05-27T00:39:44.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2583964021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc0d2c20 0x7f11cc0ce690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 -- 172.21.15.111:0/2583964021 >> 172.21.15.111:0/2583964021 conn(0x7f11cc0b5b30 msgr2=0x7f11cc0b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:44.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f4a48899640 1 Processor -- start 2024-05-27T00:39:44.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f131d349640 1 -- 172.21.15.111:0/991528140 wait complete. 2024-05-27T00:39:44.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 -- 172.21.15.111:0/2583964021 shutdown_connections 2024-05-27T00:39:44.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f11d1e53640 1 -- 172.21.15.111:0/2583964021 wait complete. 2024-05-27T00:39:44.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f2919f45640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c054430 0x7f290c0f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f291a746640 1 -- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f290c100f80 msgr2=0x7f290c0f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f291a746640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f290c100f80 0x7f290c0f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f291a746640 1 -- 172.21.15.111:0/869816511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c054430 msgr2=0x7f290c0f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f291a746640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c054430 0x7f290c0f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f291a746640 1 -- 172.21.15.111:0/869816511 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c04d0c0 con 0x7f290c19f3e0 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.813+0000 7f2919f45640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c054430 0x7f290c0f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.814+0000 7f291a746640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 0x7f290c0f9270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f290400d580 tx=0x7f290400da40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.814+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904004240 con 0x7f290c19f3e0 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.814+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29040043e0 con 0x7f290c19f3e0 2024-05-27T00:39:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904003850 con 0x7f290c19f3e0 2024-05-27T00:39:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.812+0000 7f02d0de9640 1 -- 172.21.15.111:0/2185259501 wait complete. 2024-05-27T00:39:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f02d0de9640 1 Processor -- start 2024-05-27T00:39:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f02d0de9640 1 -- start start 2024-05-27T00:39:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290c04d3a0 con 0x7f290c19f3e0 2024-05-27T00:39:44.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f4a48899640 1 -- start start 2024-05-27T00:39:44.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f11d1e53640 1 Processor -- start 2024-05-27T00:39:44.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f290c04d960 con 0x7f290c19f3e0 2024-05-27T00:39:44.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.815+0000 7f11d1e53640 1 -- start start 2024-05-27T00:39:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.816+0000 7f131d349640 1 Processor -- start 2024-05-27T00:39:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.816+0000 7f131d349640 1 -- start start 2024-05-27T00:39:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.816+0000 7f4a48899640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a3c03a380 0x7f4a3c0e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290c04db60 con 0x7f290c19f3e0 2024-05-27T00:39:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a48899640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c054670 0x7f4a3c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f02d0de9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c400dc60 0x7f02c4189130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a48899640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 0x7f4a3c181000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f02d0de9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 0x7f02c4007b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f02d0de9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c40e3610 0x7f02c4008050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a48899640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c03a8d0 con 0x7f4a3c03a380 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a48899640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c0eafa0 con 0x7f4a3c054670 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a4909a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 0x7f4a3c181000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a4909a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 0x7f4a3c181000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48118/0 (socket says 172.21.15.111:48118) 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a4909a640 1 -- 172.21.15.111:0/1121017979 learned_addr learned my addr 172.21.15.111:0/1121017979 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c03aa50 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11d1e53640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc06d510 0x7f11cc0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11d1e53640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f11cc0d2c20 0x7f11cc1ee080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11d1e53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11cc166820 0x7f11cc1ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11d1e53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc166c30 con 0x7f11cc166820 2024-05-27T00:39:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11d1e53640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0d0490 con 0x7f11cc06d510 2024-05-27T00:39:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11d1e53640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc166db0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f4a4259c640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c054670 0x7f4a3c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11cbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11cc166820 0x7f11cc1ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11cbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11cc166820 0x7f11cc1ee5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51234/0 (socket says 172.21.15.111:51234) 2024-05-27T00:39:44.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11cbfff640 1 -- 172.21.15.111:0/2302156980 learned_addr learned my addr 172.21.15.111:0/2302156980 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131d349640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13100dfef0 0x7f13100ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a42d9d640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a3c03a380 0x7f4a3c0e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131d349640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 0x7f13100e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131d349640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13100cca00 0x7f13100e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4909a640 1 -- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c054670 msgr2=0x7f4a3c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4909a640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c054670 0x7f4a3c0d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4909a640 1 -- 172.21.15.111:0/1121017979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a3c03a380 msgr2=0x7f4a3c0e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f13177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 0x7f13100e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f13177fe640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 0x7f13100e06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56658/0 (socket says 172.21.15.111:56658) 2024-05-27T00:39:44.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4909a640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a3c03a380 0x7f4a3c0e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4909a640 1 -- 172.21.15.111:0/1121017979 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c04b410 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4259c640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c054670 0x7f4a3c0d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:44.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f13177fe640 1 -- 172.21.15.111:0/2912121372 learned_addr learned my addr 172.21.15.111:0/2912121372 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131cb48640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13100cca00 0x7f13100e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a4909a640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 0x7f4a3c181000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4a44073d80 tx=0x7f4a44075870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1310056b50 con 0x7f13100cca00 2024-05-27T00:39:44.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1310009140 con 0x7f13100e4100 2024-05-27T00:39:44.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1310008b10 con 0x7f13100dfef0 2024-05-27T00:39:44.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440748f0 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44074a90 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44074ec0 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c04b6b0 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c0d2160 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.817+0000 7f11cb7fe640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc06d510 0x7f11cc0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f11cc0d2c20 0x7f11cc1ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 -- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc06d510 msgr2=0x7f11cc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc06d510 0x7f11cc0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 -- 172.21.15.111:0/2302156980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11cc166820 msgr2=0x7f11cc1ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11cc166820 0x7f11cc1ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 -- 172.21.15.111:0/2302156980 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0cb5d0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11caffd640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f11cc0d2c20 0x7f11cc1ee080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00c440 tx=0x7f11bc001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc017070 con 0x7f11cc0d2c20 2024-05-27T00:39:44.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc005ce0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0072d0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc0cb8b0 con 0x7f11cc0d2c20 2024-05-27T00:39:44.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc0cbe70 con 0x7f11cc0d2c20 2024-05-27T00:39:44.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f1317fff640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13100dfef0 0x7f13100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f13177fe640 1 -- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13100dfef0 msgr2=0x7f13100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f13177fe640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13100dfef0 0x7f13100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f13177fe640 1 -- 172.21.15.111:0/2912121372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13100cca00 msgr2=0x7f13100e0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f13177fe640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13100cca00 0x7f13100e0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f13177fe640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13100cd530 con 0x7f13100e4100 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02c9d9b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 0x7f02c4007b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02c9d9b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 0x7f02c4007b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48134/0 (socket says 172.21.15.111:48134) 2024-05-27T00:39:44.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02d0de9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02c40e4770 con 0x7f02c40e3610 2024-05-27T00:39:44.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02d0de9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f02c40e3b40 con 0x7f02c400dc60 2024-05-27T00:39:44.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02d0de9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f02c4055ae0 con 0x7f02c4035cc0 2024-05-27T00:39:44.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.820+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29040039f0 con 0x7f290c19f3e0 2024-05-27T00:39:44.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02c9d9b640 1 -- 172.21.15.111:0/2144869529 learned_addr learned my addr 172.21.15.111:0/2144869529 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:44.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02c9d9b640 1 -- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c400dc60 msgr2=0x7f02c4189130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:44.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc068b00 con 0x7f11cc0d2c20 2024-05-27T00:39:44.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02c9d9b640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c400dc60 0x7f02c4189130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.818+0000 7f02c9d9b640 1 -- 172.21.15.111:0/2144869529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c40e3610 msgr2=0x7f02c4008050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T00:39:44.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f02cad9d640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c40e3610 0x7f02c4008050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.819+0000 7f02c9d9b640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c40e3610 0x7f02c4008050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:44.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.820+0000 7f02c9d9b640 1 -- 172.21.15.111:0/2144869529 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c400a230 con 0x7f02c4035cc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.820+0000 7f02c9d9b640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 0x7f02c4007b10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b3e0 tx=0x7f02c000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0015020 con 0x7f02c4035cc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c0011830 con 0x7f02c4035cc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0011c40 con 0x7f02c4035cc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c400a510 con 0x7f02c4035cc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c4189670 con 0x7f02c4035cc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a440731c0 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f4a2b7fe640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a3006be30 0x7f4a3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f13177fe640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 0x7f13100e06c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f130c0094e0 tx=0x7f130c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c01f070 con 0x7f13100e4100 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130c013470 con 0x7f13100e4100 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c01be60 con 0x7f13100e4100 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131004a7d0 con 0x7f13100e4100 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131004aa70 con 0x7f13100e4100 2024-05-27T00:39:44.855 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4a44106340 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c404e770 con 0x7f02c4035cc0 2024-05-27T00:39:44.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f290a7fc640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f28f806bf00 0x7f28f806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.821+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f290408b360 con 0x7f290c19f3e0 2024-05-27T00:39:44.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f2919f45640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f28f806bf00 0x7f28f806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f4a42d9d640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a3006be30 0x7f4a3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c009000 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.822+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11bc01a070 con 0x7f11cc0d2c20 2024-05-27T00:39:44.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.823+0000 7f2919f45640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f28f806bf00 0x7f28f806e3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f290c008680 tx=0x7f291406f030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.823+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13100057e0 con 0x7f13100e4100 2024-05-27T00:39:44.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.823+0000 7f4a42d9d640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a3006be30 0x7f4a3006e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034000 tx=0x7f4a3802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.823+0000 7f11b3fff640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f11b807cf40 0x7f11b807f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.823+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f11bc08a440 con 0x7f11cc0d2c20 2024-05-27T00:39:44.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.823+0000 7f11cb7fe640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f11b807cf40 0x7f11b807f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.824+0000 7f11cb7fe640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f11b807cf40 0x7f11b807f400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11c4034000 tx=0x7f11c402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.825+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f130c022070 con 0x7f13100e4100 2024-05-27T00:39:44.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.825+0000 7f1314ff9640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f12fc06bf00 0x7f12fc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.825+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f130c094190 con 0x7f13100e4100 2024-05-27T00:39:44.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.825+0000 7f1317fff640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f12fc06bf00 0x7f12fc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.826+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f290c04db60 con 0x7f290c19f3e0 2024-05-27T00:39:44.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.826+0000 7f1317fff640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f12fc06bf00 0x7f12fc06e3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1304034000 tx=0x7f130402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.828+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f130c062190 con 0x7f13100e4100 2024-05-27T00:39:44.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.831+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f11bc058440 con 0x7f11cc0d2c20 2024-05-27T00:39:44.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.832+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02c0018020 con 0x7f02c4035cc0 2024-05-27T00:39:44.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.833+0000 7f02baffd640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f02ac06be80 0x7f02ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:44.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.833+0000 7f02ca59c640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f02ac06be80 0x7f02ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:44.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.834+0000 7f02ca59c640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f02ac06be80 0x7f02ac06e340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02bc034000 tx=0x7f02bc02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:44.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.835+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f02c0011de0 con 0x7f02c4035cc0 2024-05-27T00:39:44.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.835+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4a440d4910 con 0x7f4a3c0eabc0 2024-05-27T00:39:44.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:44.840+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f02c001f020 con 0x7f02c4035cc0 2024-05-27T00:39:45.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.091+0000 7f6baf379640 1 -- 172.21.15.111:0/477018077 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6b90005580 con 0x7f6ba0049850 2024-05-27T00:39:45.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.092+0000 7f6b9effd640 1 -- 172.21.15.111:0/477018077 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b9807a170 con 0x7f6ba0049850 2024-05-27T00:39:45.093 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 -- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6b8c06bf30 msgr2=0x7f6b8c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6b8c06bf30 0x7f6b8c06e3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba804fc80 tx=0x7f6ba806b2c0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 -- 172.21.15.111:0/477018077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 msgr2=0x7f6ba0045970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 0x7f6ba0045970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b98031000 tx=0x7f6b980046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 -- 172.21.15.111:0/477018077 shutdown_connections 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6b8c06bf30 0x7f6b8c06e3f0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba804fc80 tx=0x7f6ba806b2c0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ba0049850 0x7f6ba0045970 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6ba00ead00 0x7f6ba00dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.096+0000 7f6b9cff9640 1 --2- 172.21.15.111:0/477018077 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6ba004ffd0 0x7f6ba004b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.097+0000 7f6b9cff9640 1 -- 172.21.15.111:0/477018077 >> 172.21.15.111:0/477018077 conn(0x7f6ba0009fb0 msgr2=0x7f6ba00e8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.097+0000 7f6b9cff9640 1 -- 172.21.15.111:0/477018077 shutdown_connections 2024-05-27T00:39:45.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.097+0000 7f6b9cff9640 1 -- 172.21.15.111:0/477018077 wait complete. 2024-05-27T00:39:45.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.2 2024-05-27T00:39:45.115 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.146+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f469c04f1d0 con 0x7f469c054d80 2024-05-27T00:39:45.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.147+0000 7f46a17fa640 1 -- 172.21.15.111:0/534539070 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f469c04f1d0 con 0x7f469c054d80 2024-05-27T00:39:45.148 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.150+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f468806be30 msgr2=0x7f468806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.150+0000 7f46a9a43640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f468806be30 0x7f468806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46a404fc80 tx=0x7f46a40ab000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.150+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c054d80 msgr2=0x7f469c006670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.150+0000 7f46a9a43640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c054d80 0x7f469c006670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f469800b3e0 tx=0x7f469800b8a0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 shutdown_connections 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f468806be30 0x7f468806e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f469c054d80 0x7f469c006670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f469c050120 0x7f469c035750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 --2- 172.21.15.111:0/534539070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f469c0387f0 0x7f469c185ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 >> 172.21.15.111:0/534539070 conn(0x7f469c055350 msgr2=0x7f469c04c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 shutdown_connections 2024-05-27T00:39:45.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.151+0000 7f46a9a43640 1 -- 172.21.15.111:0/534539070 wait complete. 2024-05-27T00:39:45.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2024-05-27T00:39:45.168 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.170+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2c78072d80 con 0x7f2c78134730 2024-05-27T00:39:45.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.170+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f131000f3c0 con 0x7f13100e4100 2024-05-27T00:39:45.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.170+0000 7f2c74ff9640 1 -- 172.21.15.111:0/3703239377 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2c680586b0 con 0x7f2c78134730 2024-05-27T00:39:45.172 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.171+0000 7f1314ff9640 1 -- 172.21.15.111:0/2912121372 <== mon.1 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f130c066490 con 0x7f13100e4100 2024-05-27T00:39:45.172 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f12fc06bf00 msgr2=0x7f12fc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f12fc06bf00 0x7f12fc06e3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1304034000 tx=0x7f130402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 msgr2=0x7f13100e06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 0x7f13100e06c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f130c0094e0 tx=0x7f130c00ee70 comp rx=0 tx=0).stop 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 shutdown_connections 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f12fc06bf00 0x7f12fc06e3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13100cca00 0x7f13100e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13100e4100 0x7f13100e06c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 --2- 172.21.15.111:0/2912121372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f13100dfef0 0x7f13100ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 >> 172.21.15.111:0/2912121372 conn(0x7f131001ef40 msgr2=0x7f131000cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 shutdown_connections 2024-05-27T00:39:45.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f131d349640 1 -- 172.21.15.111:0/2912121372 wait complete. 2024-05-27T00:39:45.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c6406c0d0 msgr2=0x7f2c6406e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.173+0000 7f2c7d626640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c6406c0d0 0x7f2c6406e590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c034000 tx=0x7f2c6c02e000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 msgr2=0x7f2c78152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 0x7f2c78152d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800cac0 tx=0x7f2c6800cf80 comp rx=0 tx=0).stop 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 shutdown_connections 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c6406c0d0 0x7f2c6406e590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c781532a0 0x7f2c78087ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c78134730 0x7f2c78152d60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 --2- 172.21.15.111:0/3703239377 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c7814c020 0x7f2c78162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 >> 172.21.15.111:0/3703239377 conn(0x7f2c7806d300 msgr2=0x7f2c780b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 shutdown_connections 2024-05-27T00:39:45.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.174+0000 7f2c7d626640 1 -- 172.21.15.111:0/3703239377 wait complete. 2024-05-27T00:39:45.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-05-27T00:39:45.187 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-05-27T00:39:45.191 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.204+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f02c4005390 con 0x7f02c4035cc0 2024-05-27T00:39:45.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.205+0000 7f02baffd640 1 -- 172.21.15.111:0/2144869529 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02c000f2e0 con 0x7f02c4035cc0 2024-05-27T00:39:45.206 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f02ac06be80 msgr2=0x7f02ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f02ac06be80 0x7f02ac06e340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02bc034000 tx=0x7f02bc02c000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 msgr2=0x7f02c4007b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 0x7f02c4007b10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b3e0 tx=0x7f02c000b8a0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 shutdown_connections 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f02ac06be80 0x7f02ac06e340 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02c40e3610 0x7f02c4008050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f02c4035cc0 0x7f02c4007b10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 --2- 172.21.15.111:0/2144869529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c400dc60 0x7f02c4189130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 >> 172.21.15.111:0/2144869529 conn(0x7f02c4056ce0 msgr2=0x7f02c4055ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 shutdown_connections 2024-05-27T00:39:45.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.207+0000 7f02d0de9640 1 -- 172.21.15.111:0/2144869529 wait complete. 2024-05-27T00:39:45.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.212+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f11cc0d0000 con 0x7f11cc0d2c20 2024-05-27T00:39:45.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.213+0000 7f11b3fff640 1 -- 172.21.15.111:0/2302156980 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f11bc05c740 con 0x7f11cc0d2c20 2024-05-27T00:39:45.214 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f11b807cf40 msgr2=0x7f11b807f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f11b807cf40 0x7f11b807f400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11c4034000 tx=0x7f11c402f000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f11cc0d2c20 msgr2=0x7f11cc1ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f11cc0d2c20 0x7f11cc1ee080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00c440 tx=0x7f11bc001e70 comp rx=0 tx=0).stop 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 shutdown_connections 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f11b807cf40 0x7f11b807f400 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11cc166820 0x7f11cc1ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f11cc0d2c20 0x7f11cc1ee080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 --2- 172.21.15.111:0/2302156980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11cc06d510 0x7f11cc0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 >> 172.21.15.111:0/2302156980 conn(0x7f11cc0b5b30 msgr2=0x7f11cc162900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 shutdown_connections 2024-05-27T00:39:45.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.215+0000 7f11d1e53640 1 -- 172.21.15.111:0/2302156980 wait complete. 2024-05-27T00:39:45.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.7 2024-05-27T00:39:45.219 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2024-05-27T00:39:45.227 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.239+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f290c0f5ab0 con 0x7f290c19f3e0 2024-05-27T00:39:45.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.240+0000 7f290a7fc640 1 -- 172.21.15.111:0/869816511 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29040592e0 con 0x7f290c19f3e0 2024-05-27T00:39:45.240 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.242+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f28f806bf00 msgr2=0x7f28f806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.242+0000 7f291af47640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f28f806bf00 0x7f28f806e3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f290c008680 tx=0x7f291406f030 comp rx=0 tx=0).stop 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.242+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 msgr2=0x7f290c0f9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.242+0000 7f291af47640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 0x7f290c0f9270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f290400d580 tx=0x7f290400da40 comp rx=0 tx=0).stop 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 shutdown_connections 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f28f806bf00 0x7f28f806e3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f290c19f3e0 0x7f290c0f9270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f290c100f80 0x7f290c0f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 --2- 172.21.15.111:0/869816511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f290c054430 0x7f290c0f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 >> 172.21.15.111:0/869816511 conn(0x7f290c009fb0 msgr2=0x7f290c0057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 shutdown_connections 2024-05-27T00:39:45.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.243+0000 7f291af47640 1 -- 172.21.15.111:0/869816511 wait complete. 2024-05-27T00:39:45.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.255+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4a3c0059c0 con 0x7f4a3c0eabc0 2024-05-27T00:39:45.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.255+0000 7f4a2b7fe640 1 -- 172.21.15.111:0/1121017979 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4a440d4210 con 0x7f4a3c0eabc0 2024-05-27T00:39:45.256 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2024-05-27T00:39:45.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.4 2024-05-27T00:39:45.258 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a3006be30 msgr2=0x7f4a3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a3006be30 0x7f4a3006e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034000 tx=0x7f4a3802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 msgr2=0x7f4a3c181000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 0x7f4a3c181000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4a44073d80 tx=0x7f4a44075870 comp rx=0 tx=0).stop 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 shutdown_connections 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a3006be30 0x7f4a3006e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a3c0eabc0 0x7f4a3c181000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a3c054670 0x7f4a3c0d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 --2- 172.21.15.111:0/1121017979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a3c03a380 0x7f4a3c0e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 >> 172.21.15.111:0/1121017979 conn(0x7f4a3c0550f0 msgr2=0x7f4a3c007070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 shutdown_connections 2024-05-27T00:39:45.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.258+0000 7f4a48899640 1 -- 172.21.15.111:0/1121017979 wait complete. 2024-05-27T00:39:45.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-05-27T00:39:45.269 DEBUG:teuthology.parallel:result is None 2024-05-27T00:39:45.269 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-27T00:39:45.270 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:39:45.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.408+0000 7f0c078c3640 1 Processor -- start 2024-05-27T00:39:45.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.408+0000 7f0c078c3640 1 -- start start 2024-05-27T00:39:45.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c078c3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c0006b390 0x7f0c000cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c078c3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c000b36e0 con 0x7f0c000cdc80 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c078c3640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c000b39c0 con 0x7f0c00144000 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c078c3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000b42f0 con 0x7f0c0006b390 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c05e60640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0c000cdc80 0x7f0c000726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36740/0 (socket says 172.21.15.111:36740) 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c05e60640 1 -- 172.21.15.111:0/1099455554 learned_addr learned my addr 172.21.15.111:0/1099455554 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c04e5e640 1 --2- 172.21.15.111:0/1099455554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c0006b390 0x7f0c000cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085566521 0 0) 0x7f0c000b36e0 con 0x7f0c000cdc80 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be8005730 con 0x7f0c000cdc80 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356928629 0 0) 0x7f0c000b39c0 con 0x7f0c00144000 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c000b36e0 con 0x7f0c00144000 2024-05-27T00:39:45.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c04e5e640 1 -- 172.21.15.111:0/1099455554 >> v1:172.21.15.153:6789/0 conn(0x7f0c00144000 legacy=0x7f0c000b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c04e5e640 1 -- 172.21.15.111:0/1099455554 >> v1:172.21.15.111:6789/0 conn(0x7f0c000cdc80 legacy=0x7f0c000726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.409+0000 7f0c04e5e640 1 -- 172.21.15.111:0/1099455554 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c000b3ca0 con 0x7f0c0006b390 2024-05-27T00:39:45.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.410+0000 7f0c04e5e640 1 --2- 172.21.15.111:0/1099455554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c0006b390 0x7f0c000cd740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40097d0 tx=0x7f0bf402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199a8671f9778ede server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.410+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4037070 con 0x7f0c0006b390 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.410+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf4032440 con 0x7f0c0006b390 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.410+0000 7f0bfffff640 1 -- 172.21.15.111:0/1099455554 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4032810 con 0x7f0c0006b390 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 -- 172.21.15.111:0/1099455554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c0006b390 msgr2=0x7f0c000cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1099455554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c0006b390 0x7f0c000cd740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40097d0 tx=0x7f0bf402a210 comp rx=0 tx=0).stop 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 -- 172.21.15.111:0/1099455554 shutdown_connections 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1099455554 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c0006b390 0x7f0c000cd740 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 -- 172.21.15.111:0/1099455554 >> 172.21.15.111:0/1099455554 conn(0x7f0c000b1c00 msgr2=0x7f0c000b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 -- 172.21.15.111:0/1099455554 shutdown_connections 2024-05-27T00:39:45.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.411+0000 7f0c078c3640 1 -- 172.21.15.111:0/1099455554 wait complete. 2024-05-27T00:39:45.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.412+0000 7f0c078c3640 1 Processor -- start 2024-05-27T00:39:45.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.412+0000 7f0c078c3640 1 -- start start 2024-05-27T00:39:45.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c078c3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 0x7f0c0013f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c078c3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c000cdc80 0x7f0c0013fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c078c3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c00144000 0x7f0c0014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c078c3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00072f60 con 0x7f0c0006b390 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c078c3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0c00072de0 con 0x7f0c000cdc80 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c078c3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000b4780 con 0x7f0c00144000 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 0x7f0c0013f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 0x7f0c0013f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51250/0 (socket says 172.21.15.111:51250) 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 -- 172.21.15.111:0/1734372676 learned_addr learned my addr 172.21.15.111:0/1734372676 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c04e5e640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c000cdc80 0x7f0c0013fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c05e60640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c00144000 0x7f0c0014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 -- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c00144000 msgr2=0x7f0c0014e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c00144000 0x7f0c0014e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 -- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c000cdc80 msgr2=0x7f0c0013fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c000cdc80 0x7f0c0013fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c0565f640 1 -- 172.21.15.111:0/1734372676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c000ada60 con 0x7f0c0006b390 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.413+0000 7f0c05e60640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c00144000 0x7f0c0014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.414+0000 7f0c0565f640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 0x7f0c0013f780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c430 tx=0x7f0bf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.414+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0017070 con 0x7f0c0006b390 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.414+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf00052f0 con 0x7f0c0006b390 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.414+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c000add40 con 0x7f0c0006b390 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.414+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0003830 con 0x7f0c0006b390 2024-05-27T00:39:45.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.414+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c000ae300 con 0x7f0c0006b390 2024-05-27T00:39:45.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.415+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c00068680 con 0x7f0c0006b390 2024-05-27T00:39:45.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.416+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bf001a070 con 0x7f0c0006b390 2024-05-27T00:39:45.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.416+0000 7f0bfdffb640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0bec06be30 0x7f0bec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.416+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0bf008a640 con 0x7f0c0006b390 2024-05-27T00:39:45.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.416+0000 7f0c04e5e640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0bec06be30 0x7f0bec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.417+0000 7f0c04e5e640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0bec06be30 0x7f0bec06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4034000 tx=0x7f0bf402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:45.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.420+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0bf0058c30 con 0x7f0c0006b390 2024-05-27T00:39:45.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.591+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c000ad810 con 0x7f0bec06be30 2024-05-27T00:39:45.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.601+0000 7f0bfdffb640 1 -- 172.21.15.111:0/1734372676 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f0c000ad810 con 0x7f0bec06be30 2024-05-27T00:39:45.606 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:45.608 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:39:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0bec06be30 msgr2=0x7f0bec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0bec06be30 0x7f0bec06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4034000 tx=0x7f0bf402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 msgr2=0x7f0c0013f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 0x7f0c0013f780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c430 tx=0x7f0bf000c8f0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 shutdown_connections 2024-05-27T00:39:45.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0bec06be30 0x7f0bec06e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0c00144000 0x7f0c0014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c000cdc80 0x7f0c0013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 --2- 172.21.15.111:0/1734372676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c0006b390 0x7f0c0013f780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 >> 172.21.15.111:0/1734372676 conn(0x7f0c000b1c00 msgr2=0x7f0c000af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 shutdown_connections 2024-05-27T00:39:45.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.604+0000 7f0c078c3640 1 -- 172.21.15.111:0/1734372676 wait complete. 2024-05-27T00:39:45.616 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-27T00:39:44.930364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003519"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133844+0000","last_change":"2024-05-27T00:39:20.574932+0000","last_active":"2024-05-27T00:39:28.133844+0000","last_peered":"2024-05-27T00:39:28.133844+0000","last_clean":"2024-05-27T00:39:28.133844+0000","last_became_active":"2024-05-27T00:39:20.574617+0000","last_became_peered":"2024-05-27T00:39:20.574617+0000","last_unstale":"2024-05-27T00:39:28.133844+0000","last_undegraded":"2024-05-27T00:39:28.133844+0000","last_fullsized":"2024-05-27T00:39:28.133844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610219+0000","last_change":"2024-05-27T00:39:21.594550+0000","last_active":"2024-05-27T00:39:27.610219+0000","last_peered":"2024-05-27T00:39:27.610219+0000","last_clean":"2024-05-27T00:39:27.610219+0000","last_became_active":"2024-05-27T00:39:21.594296+0000","last_became_peered":"2024-05-27T00:39:21.594296+0000","last_unstale":"2024-05-27T00:39:27.610219+0000","last_undegraded":"2024-05-27T00:39:27.610219+0000","last_fullsized":"2024-05-27T00:39:27.610219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609737+0000","last_change":"2024-05-27T00:39:25.965194+0000","last_active":"2024-05-27T00:39:27.609737+0000","last_peered":"2024-05-27T00:39:27.609737+0000","last_clean":"2024-05-27T00:39:27.609737+0000","last_became_active":"2024-05-27T00:39:25.964829+0000","last_became_peered":"2024-05-27T00:39:25.964829+0000","last_unstale":"2024-05-27T00:39:27.609737+0000","last_undegraded":"2024-05-27T00:39:27.609737+0000","last_fullsized":"2024-05-27T00:39:27.609737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:38.727603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610308+0000","last_change":"2024-05-27T00:39:20.569739+0000","last_active":"2024-05-27T00:39:27.610308+0000","last_peered":"2024-05-27T00:39:27.610308+0000","last_clean":"2024-05-27T00:39:27.610308+0000","last_became_active":"2024-05-27T00:39:20.569481+0000","last_became_peered":"2024-05-27T00:39:20.569481+0000","last_unstale":"2024-05-27T00:39:27.610308+0000","last_undegraded":"2024-05-27T00:39:27.610308+0000","last_fullsized":"2024-05-27T00:39:27.610308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133753+0000","last_change":"2024-05-27T00:39:21.576759+0000","last_active":"2024-05-27T00:39:28.133753+0000","last_peered":"2024-05-27T00:39:28.133753+0000","last_clean":"2024-05-27T00:39:28.133753+0000","last_became_active":"2024-05-27T00:39:21.576492+0000","last_became_peered":"2024-05-27T00:39:21.576492+0000","last_unstale":"2024-05-27T00:39:28.133753+0000","last_undegraded":"2024-05-27T00:39:28.133753+0000","last_fullsized":"2024-05-27T00:39:28.133753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833532+0000","last_change":"2024-05-27T00:39:25.626523+0000","last_active":"2024-05-27T00:39:28.833532+0000","last_peered":"2024-05-27T00:39:28.833532+0000","last_clean":"2024-05-27T00:39:28.833532+0000","last_became_active":"2024-05-27T00:39:25.626132+0000","last_became_peered":"2024-05-27T00:39:25.626132+0000","last_unstale":"2024-05-27T00:39:28.833532+0000","last_undegraded":"2024-05-27T00:39:28.833532+0000","last_fullsized":"2024-05-27T00:39:28.833532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834538+0000","last_change":"2024-05-27T00:39:20.577889+0000","last_active":"2024-05-27T00:39:28.834538+0000","last_peered":"2024-05-27T00:39:28.834538+0000","last_clean":"2024-05-27T00:39:28.834538+0000","last_became_active":"2024-05-27T00:39:20.577565+0000","last_became_peered":"2024-05-27T00:39:20.577565+0000","last_unstale":"2024-05-27T00:39:28.834538+0000","last_undegraded":"2024-05-27T00:39:28.834538+0000","last_fullsized":"2024-05-27T00:39:28.834538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:12:22.836110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610522+0000","last_change":"2024-05-27T00:39:21.580044+0000","last_active":"2024-05-27T00:39:27.610522+0000","last_peered":"2024-05-27T00:39:27.610522+0000","last_clean":"2024-05-27T00:39:27.610522+0000","last_became_active":"2024-05-27T00:39:21.579770+0000","last_became_peered":"2024-05-27T00:39:21.579770+0000","last_unstale":"2024-05-27T00:39:27.610522+0000","last_undegraded":"2024-05-27T00:39:27.610522+0000","last_fullsized":"2024-05-27T00:39:27.610522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833971+0000","last_change":"2024-05-27T00:39:25.627365+0000","last_active":"2024-05-27T00:39:28.833971+0000","last_peered":"2024-05-27T00:39:28.833971+0000","last_clean":"2024-05-27T00:39:28.833971+0000","last_became_active":"2024-05-27T00:39:25.626973+0000","last_became_peered":"2024-05-27T00:39:25.626973+0000","last_unstale":"2024-05-27T00:39:28.833971+0000","last_undegraded":"2024-05-27T00:39:28.833971+0000","last_fullsized":"2024-05-27T00:39:28.833971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833143+0000","last_change":"2024-05-27T00:39:20.577815+0000","last_active":"2024-05-27T00:39:28.833143+0000","last_peered":"2024-05-27T00:39:28.833143+0000","last_clean":"2024-05-27T00:39:28.833143+0000","last_became_active":"2024-05-27T00:39:20.577495+0000","last_became_peered":"2024-05-27T00:39:20.577495+0000","last_unstale":"2024-05-27T00:39:28.833143+0000","last_undegraded":"2024-05-27T00:39:28.833143+0000","last_fullsized":"2024-05-27T00:39:28.833143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:05.446553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610465+0000","last_change":"2024-05-27T00:39:21.576226+0000","last_active":"2024-05-27T00:39:27.610465+0000","last_peered":"2024-05-27T00:39:27.610465+0000","last_clean":"2024-05-27T00:39:27.610465+0000","last_became_active":"2024-05-27T00:39:21.575940+0000","last_became_peered":"2024-05-27T00:39:21.575940+0000","last_unstale":"2024-05-27T00:39:27.610465+0000","last_undegraded":"2024-05-27T00:39:27.610465+0000","last_fullsized":"2024-05-27T00:39:27.610465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833014+0000","last_change":"2024-05-27T00:39:25.623493+0000","last_active":"2024-05-27T00:39:28.833014+0000","last_peered":"2024-05-27T00:39:28.833014+0000","last_clean":"2024-05-27T00:39:28.833014+0000","last_became_active":"2024-05-27T00:39:25.623101+0000","last_became_peered":"2024-05-27T00:39:25.623101+0000","last_unstale":"2024-05-27T00:39:28.833014+0000","last_undegraded":"2024-05-27T00:39:28.833014+0000","last_fullsized":"2024-05-27T00:39:28.833014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:12.831089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834471+0000","last_change":"2024-05-27T00:39:21.594839+0000","last_active":"2024-05-27T00:39:28.834471+0000","last_peered":"2024-05-27T00:39:28.834471+0000","last_clean":"2024-05-27T00:39:28.834471+0000","last_became_active":"2024-05-27T00:39:21.594428+0000","last_became_peered":"2024-05-27T00:39:21.594428+0000","last_unstale":"2024-05-27T00:39:28.834471+0000","last_undegraded":"2024-05-27T00:39:28.834471+0000","last_fullsized":"2024-05-27T00:39:28.834471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833832+0000","last_change":"2024-05-27T00:39:20.579815+0000","last_active":"2024-05-27T00:39:28.833832+0000","last_peered":"2024-05-27T00:39:28.833832+0000","last_clean":"2024-05-27T00:39:28.833832+0000","last_became_active":"2024-05-27T00:39:20.579507+0000","last_became_peered":"2024-05-27T00:39:20.579507+0000","last_unstale":"2024-05-27T00:39:28.833832+0000","last_undegraded":"2024-05-27T00:39:28.833832+0000","last_fullsized":"2024-05-27T00:39:28.833832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:17.368730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833761+0000","last_change":"2024-05-27T00:39:25.625823+0000","last_active":"2024-05-27T00:39:28.833761+0000","last_peered":"2024-05-27T00:39:28.833761+0000","last_clean":"2024-05-27T00:39:28.833761+0000","last_became_active":"2024-05-27T00:39:25.625378+0000","last_became_peered":"2024-05-27T00:39:25.625378+0000","last_unstale":"2024-05-27T00:39:28.833761+0000","last_undegraded":"2024-05-27T00:39:28.833761+0000","last_fullsized":"2024-05-27T00:39:28.833761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609891+0000","last_change":"2024-05-27T00:39:21.591774+0000","last_active":"2024-05-27T00:39:27.609891+0000","last_peered":"2024-05-27T00:39:27.609891+0000","last_clean":"2024-05-27T00:39:27.609891+0000","last_became_active":"2024-05-27T00:39:21.591413+0000","last_became_peered":"2024-05-27T00:39:21.591413+0000","last_unstale":"2024-05-27T00:39:27.609891+0000","last_undegraded":"2024-05-27T00:39:27.609891+0000","last_fullsized":"2024-05-27T00:39:27.609891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604467+0000","last_change":"2024-05-27T00:39:20.562227+0000","last_active":"2024-05-27T00:39:27.604467+0000","last_peered":"2024-05-27T00:39:27.604467+0000","last_clean":"2024-05-27T00:39:27.604467+0000","last_became_active":"2024-05-27T00:39:20.561938+0000","last_became_peered":"2024-05-27T00:39:20.561938+0000","last_unstale":"2024-05-27T00:39:27.604467+0000","last_undegraded":"2024-05-27T00:39:27.604467+0000","last_fullsized":"2024-05-27T00:39:27.604467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834385+0000","last_change":"2024-05-27T00:39:25.628730+0000","last_active":"2024-05-27T00:39:28.834385+0000","last_peered":"2024-05-27T00:39:28.834385+0000","last_clean":"2024-05-27T00:39:28.834385+0000","last_became_active":"2024-05-27T00:39:25.628352+0000","last_became_peered":"2024-05-27T00:39:25.628352+0000","last_unstale":"2024-05-27T00:39:28.834385+0000","last_undegraded":"2024-05-27T00:39:28.834385+0000","last_fullsized":"2024-05-27T00:39:28.834385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609789+0000","last_change":"2024-05-27T00:39:21.585438+0000","last_active":"2024-05-27T00:39:27.609789+0000","last_peered":"2024-05-27T00:39:27.609789+0000","last_clean":"2024-05-27T00:39:27.609789+0000","last_became_active":"2024-05-27T00:39:21.585062+0000","last_became_peered":"2024-05-27T00:39:21.585062+0000","last_unstale":"2024-05-27T00:39:27.609789+0000","last_undegraded":"2024-05-27T00:39:27.609789+0000","last_fullsized":"2024-05-27T00:39:27.609789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604165+0000","last_change":"2024-05-27T00:39:20.570857+0000","last_active":"2024-05-27T00:39:27.604165+0000","last_peered":"2024-05-27T00:39:27.604165+0000","last_clean":"2024-05-27T00:39:27.604165+0000","last_became_active":"2024-05-27T00:39:20.570564+0000","last_became_peered":"2024-05-27T00:39:20.570564+0000","last_unstale":"2024-05-27T00:39:27.604165+0000","last_undegraded":"2024-05-27T00:39:27.604165+0000","last_fullsized":"2024-05-27T00:39:27.604165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605057+0000","last_change":"2024-05-27T00:39:25.963988+0000","last_active":"2024-05-27T00:39:27.605057+0000","last_peered":"2024-05-27T00:39:27.605057+0000","last_clean":"2024-05-27T00:39:27.605057+0000","last_became_active":"2024-05-27T00:39:25.963532+0000","last_became_peered":"2024-05-27T00:39:25.963532+0000","last_unstale":"2024-05-27T00:39:27.605057+0000","last_undegraded":"2024-05-27T00:39:27.605057+0000","last_fullsized":"2024-05-27T00:39:27.605057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833654+0000","last_change":"2024-05-27T00:39:21.590869+0000","last_active":"2024-05-27T00:39:28.833654+0000","last_peered":"2024-05-27T00:39:28.833654+0000","last_clean":"2024-05-27T00:39:28.833654+0000","last_became_active":"2024-05-27T00:39:21.584136+0000","last_became_peered":"2024-05-27T00:39:21.584136+0000","last_unstale":"2024-05-27T00:39:28.833654+0000","last_undegraded":"2024-05-27T00:39:28.833654+0000","last_fullsized":"2024-05-27T00:39:28.833654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:48.264046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832474+0000","last_change":"2024-05-27T00:39:20.579097+0000","last_active":"2024-05-27T00:39:28.832474+0000","last_peered":"2024-05-27T00:39:28.832474+0000","last_clean":"2024-05-27T00:39:28.832474+0000","last_became_active":"2024-05-27T00:39:20.578796+0000","last_became_peered":"2024-05-27T00:39:20.578796+0000","last_unstale":"2024-05-27T00:39:28.832474+0000","last_undegraded":"2024-05-27T00:39:28.832474+0000","last_fullsized":"2024-05-27T00:39:28.832474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833500+0000","last_change":"2024-05-27T00:39:25.628322+0000","last_active":"2024-05-27T00:39:28.833500+0000","last_peered":"2024-05-27T00:39:28.833500+0000","last_clean":"2024-05-27T00:39:28.833500+0000","last_became_active":"2024-05-27T00:39:25.627878+0000","last_became_peered":"2024-05-27T00:39:25.627878+0000","last_unstale":"2024-05-27T00:39:28.833500+0000","last_undegraded":"2024-05-27T00:39:28.833500+0000","last_fullsized":"2024-05-27T00:39:28.833500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:42.049691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133677+0000","last_change":"2024-05-27T00:39:21.581752+0000","last_active":"2024-05-27T00:39:28.133677+0000","last_peered":"2024-05-27T00:39:28.133677+0000","last_clean":"2024-05-27T00:39:28.133677+0000","last_became_active":"2024-05-27T00:39:21.581488+0000","last_became_peered":"2024-05-27T00:39:21.581488+0000","last_unstale":"2024-05-27T00:39:28.133677+0000","last_undegraded":"2024-05-27T00:39:28.133677+0000","last_fullsized":"2024-05-27T00:39:28.133677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835187+0000","last_change":"2024-05-27T00:39:20.578119+0000","last_active":"2024-05-27T00:39:28.835187+0000","last_peered":"2024-05-27T00:39:28.835187+0000","last_clean":"2024-05-27T00:39:28.835187+0000","last_became_active":"2024-05-27T00:39:20.577830+0000","last_became_peered":"2024-05-27T00:39:20.577830+0000","last_unstale":"2024-05-27T00:39:28.835187+0000","last_undegraded":"2024-05-27T00:39:28.835187+0000","last_fullsized":"2024-05-27T00:39:28.835187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832957+0000","last_change":"2024-05-27T00:39:25.965993+0000","last_active":"2024-05-27T00:39:28.832957+0000","last_peered":"2024-05-27T00:39:28.832957+0000","last_clean":"2024-05-27T00:39:28.832957+0000","last_became_active":"2024-05-27T00:39:25.965384+0000","last_became_peered":"2024-05-27T00:39:25.965384+0000","last_unstale":"2024-05-27T00:39:28.832957+0000","last_undegraded":"2024-05-27T00:39:28.832957+0000","last_fullsized":"2024-05-27T00:39:28.832957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610354+0000","last_change":"2024-05-27T00:39:21.591396+0000","last_active":"2024-05-27T00:39:27.610354+0000","last_peered":"2024-05-27T00:39:27.610354+0000","last_clean":"2024-05-27T00:39:27.610354+0000","last_became_active":"2024-05-27T00:39:21.591116+0000","last_became_peered":"2024-05-27T00:39:21.591116+0000","last_unstale":"2024-05-27T00:39:27.610354+0000","last_undegraded":"2024-05-27T00:39:27.610354+0000","last_fullsized":"2024-05-27T00:39:27.610354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604797+0000","last_change":"2024-05-27T00:39:20.573838+0000","last_active":"2024-05-27T00:39:27.604797+0000","last_peered":"2024-05-27T00:39:27.604797+0000","last_clean":"2024-05-27T00:39:27.604797+0000","last_became_active":"2024-05-27T00:39:20.573515+0000","last_became_peered":"2024-05-27T00:39:20.573515+0000","last_unstale":"2024-05-27T00:39:27.604797+0000","last_undegraded":"2024-05-27T00:39:27.604797+0000","last_fullsized":"2024-05-27T00:39:27.604797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609614+0000","last_change":"2024-05-27T00:39:25.960920+0000","last_active":"2024-05-27T00:39:27.609614+0000","last_peered":"2024-05-27T00:39:27.609614+0000","last_clean":"2024-05-27T00:39:27.609614+0000","last_became_active":"2024-05-27T00:39:25.960487+0000","last_became_peered":"2024-05-27T00:39:25.960487+0000","last_unstale":"2024-05-27T00:39:27.609614+0000","last_undegraded":"2024-05-27T00:39:27.609614+0000","last_fullsized":"2024-05-27T00:39:27.609614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:59.647950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604641+0000","last_change":"2024-05-27T00:39:21.583980+0000","last_active":"2024-05-27T00:39:27.604641+0000","last_peered":"2024-05-27T00:39:27.604641+0000","last_clean":"2024-05-27T00:39:27.604641+0000","last_became_active":"2024-05-27T00:39:21.583700+0000","last_became_peered":"2024-05-27T00:39:21.583700+0000","last_unstale":"2024-05-27T00:39:27.604641+0000","last_undegraded":"2024-05-27T00:39:27.604641+0000","last_fullsized":"2024-05-27T00:39:27.604641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834010+0000","last_change":"2024-05-27T00:39:20.579198+0000","last_active":"2024-05-27T00:39:28.834010+0000","last_peered":"2024-05-27T00:39:28.834010+0000","last_clean":"2024-05-27T00:39:28.834010+0000","last_became_active":"2024-05-27T00:39:20.578902+0000","last_became_peered":"2024-05-27T00:39:20.578902+0000","last_unstale":"2024-05-27T00:39:28.834010+0000","last_undegraded":"2024-05-27T00:39:28.834010+0000","last_fullsized":"2024-05-27T00:39:28.834010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.610865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.962036+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.961152+0000","last_became_peered":"2024-05-27T00:39:25.961152+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610557+0000","last_change":"2024-05-27T00:39:21.591906+0000","last_active":"2024-05-27T00:39:27.610557+0000","last_peered":"2024-05-27T00:39:27.610557+0000","last_clean":"2024-05-27T00:39:27.610557+0000","last_became_active":"2024-05-27T00:39:21.591611+0000","last_became_peered":"2024-05-27T00:39:21.591611+0000","last_unstale":"2024-05-27T00:39:27.610557+0000","last_undegraded":"2024-05-27T00:39:27.610557+0000","last_fullsized":"2024-05-27T00:39:27.610557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604735+0000","last_change":"2024-05-27T00:39:20.569834+0000","last_active":"2024-05-27T00:39:27.604735+0000","last_peered":"2024-05-27T00:39:27.604735+0000","last_clean":"2024-05-27T00:39:27.604735+0000","last_became_active":"2024-05-27T00:39:20.569540+0000","last_became_peered":"2024-05-27T00:39:20.569540+0000","last_unstale":"2024-05-27T00:39:27.604735+0000","last_undegraded":"2024-05-27T00:39:27.604735+0000","last_fullsized":"2024-05-27T00:39:27.604735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604262+0000","last_change":"2024-05-27T00:39:25.627533+0000","last_active":"2024-05-27T00:39:27.604262+0000","last_peered":"2024-05-27T00:39:27.604262+0000","last_clean":"2024-05-27T00:39:27.604262+0000","last_became_active":"2024-05-27T00:39:25.625872+0000","last_became_peered":"2024-05-27T00:39:25.625872+0000","last_unstale":"2024-05-27T00:39:27.604262+0000","last_undegraded":"2024-05-27T00:39:27.604262+0000","last_fullsized":"2024-05-27T00:39:27.604262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834370+0000","last_change":"2024-05-27T00:39:21.598984+0000","last_active":"2024-05-27T00:39:28.834370+0000","last_peered":"2024-05-27T00:39:28.834370+0000","last_clean":"2024-05-27T00:39:28.834370+0000","last_became_active":"2024-05-27T00:39:21.594222+0000","last_became_peered":"2024-05-27T00:39:21.594222+0000","last_unstale":"2024-05-27T00:39:28.834370+0000","last_undegraded":"2024-05-27T00:39:28.834370+0000","last_fullsized":"2024-05-27T00:39:28.834370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610187+0000","last_change":"2024-05-27T00:39:20.565350+0000","last_active":"2024-05-27T00:39:27.610187+0000","last_peered":"2024-05-27T00:39:27.610187+0000","last_clean":"2024-05-27T00:39:27.610187+0000","last_became_active":"2024-05-27T00:39:20.562112+0000","last_became_peered":"2024-05-27T00:39:20.562112+0000","last_unstale":"2024-05-27T00:39:27.610187+0000","last_undegraded":"2024-05-27T00:39:27.610187+0000","last_fullsized":"2024-05-27T00:39:27.610187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604681+0000","last_change":"2024-05-27T00:39:25.624845+0000","last_active":"2024-05-27T00:39:27.604681+0000","last_peered":"2024-05-27T00:39:27.604681+0000","last_clean":"2024-05-27T00:39:27.604681+0000","last_became_active":"2024-05-27T00:39:25.624336+0000","last_became_peered":"2024-05-27T00:39:25.624336+0000","last_unstale":"2024-05-27T00:39:27.604681+0000","last_undegraded":"2024-05-27T00:39:27.604681+0000","last_fullsized":"2024-05-27T00:39:27.604681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604350+0000","last_change":"2024-05-27T00:39:21.586786+0000","last_active":"2024-05-27T00:39:27.604350+0000","last_peered":"2024-05-27T00:39:27.604350+0000","last_clean":"2024-05-27T00:39:27.604350+0000","last_became_active":"2024-05-27T00:39:21.586521+0000","last_became_peered":"2024-05-27T00:39:21.586521+0000","last_unstale":"2024-05-27T00:39:27.604350+0000","last_undegraded":"2024-05-27T00:39:27.604350+0000","last_fullsized":"2024-05-27T00:39:27.604350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573104+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.572665+0000","last_became_peered":"2024-05-27T00:39:20.572665+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605183+0000","last_change":"2024-05-27T00:39:25.625105+0000","last_active":"2024-05-27T00:39:27.605183+0000","last_peered":"2024-05-27T00:39:27.605183+0000","last_clean":"2024-05-27T00:39:27.605183+0000","last_became_active":"2024-05-27T00:39:25.624400+0000","last_became_peered":"2024-05-27T00:39:25.624400+0000","last_unstale":"2024-05-27T00:39:27.605183+0000","last_undegraded":"2024-05-27T00:39:27.605183+0000","last_fullsized":"2024-05-27T00:39:27.605183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133189+0000","last_change":"2024-05-27T00:39:21.578555+0000","last_active":"2024-05-27T00:39:28.133189+0000","last_peered":"2024-05-27T00:39:28.133189+0000","last_clean":"2024-05-27T00:39:28.133189+0000","last_became_active":"2024-05-27T00:39:21.578291+0000","last_became_peered":"2024-05-27T00:39:21.578291+0000","last_unstale":"2024-05-27T00:39:28.133189+0000","last_undegraded":"2024-05-27T00:39:28.133189+0000","last_fullsized":"2024-05-27T00:39:28.133189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:00.616291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832328+0000","last_change":"2024-05-27T00:39:20.578837+0000","last_active":"2024-05-27T00:39:28.832328+0000","last_peered":"2024-05-27T00:39:28.832328+0000","last_clean":"2024-05-27T00:39:28.832328+0000","last_became_active":"2024-05-27T00:39:20.578414+0000","last_became_peered":"2024-05-27T00:39:20.578414+0000","last_unstale":"2024-05-27T00:39:28.832328+0000","last_undegraded":"2024-05-27T00:39:28.832328+0000","last_fullsized":"2024-05-27T00:39:28.832328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133680+0000","last_change":"2024-05-27T00:39:25.962038+0000","last_active":"2024-05-27T00:39:28.133680+0000","last_peered":"2024-05-27T00:39:28.133680+0000","last_clean":"2024-05-27T00:39:28.133680+0000","last_became_active":"2024-05-27T00:39:25.961489+0000","last_became_peered":"2024-05-27T00:39:25.961489+0000","last_unstale":"2024-05-27T00:39:28.133680+0000","last_undegraded":"2024-05-27T00:39:28.133680+0000","last_fullsized":"2024-05-27T00:39:28.133680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:36.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609779+0000","last_change":"2024-05-27T00:39:21.593420+0000","last_active":"2024-05-27T00:39:27.609779+0000","last_peered":"2024-05-27T00:39:27.609779+0000","last_clean":"2024-05-27T00:39:27.609779+0000","last_became_active":"2024-05-27T00:39:21.593165+0000","last_became_peered":"2024-05-27T00:39:21.593165+0000","last_unstale":"2024-05-27T00:39:27.609779+0000","last_undegraded":"2024-05-27T00:39:27.609779+0000","last_fullsized":"2024-05-27T00:39:27.609779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610340+0000","last_change":"2024-05-27T00:39:20.573661+0000","last_active":"2024-05-27T00:39:27.610340+0000","last_peered":"2024-05-27T00:39:27.610340+0000","last_clean":"2024-05-27T00:39:27.610340+0000","last_became_active":"2024-05-27T00:39:20.572675+0000","last_became_peered":"2024-05-27T00:39:20.572675+0000","last_unstale":"2024-05-27T00:39:27.610340+0000","last_undegraded":"2024-05-27T00:39:27.610340+0000","last_fullsized":"2024-05-27T00:39:27.610340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832965+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:39:28.832965+0000","last_peered":"2024-05-27T00:39:28.832965+0000","last_clean":"2024-05-27T00:39:28.832965+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:39:28.832965+0000","last_undegraded":"2024-05-27T00:39:28.832965+0000","last_fullsized":"2024-05-27T00:39:28.832965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834052+0000","last_change":"2024-05-27T00:39:25.964404+0000","last_active":"2024-05-27T00:39:28.834052+0000","last_peered":"2024-05-27T00:39:28.834052+0000","last_clean":"2024-05-27T00:39:28.834052+0000","last_became_active":"2024-05-27T00:39:25.963955+0000","last_became_peered":"2024-05-27T00:39:25.963955+0000","last_unstale":"2024-05-27T00:39:28.834052+0000","last_undegraded":"2024-05-27T00:39:28.834052+0000","last_fullsized":"2024-05-27T00:39:28.834052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:48.698768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834325+0000","last_change":"2024-05-27T00:39:21.585811+0000","last_active":"2024-05-27T00:39:28.834325+0000","last_peered":"2024-05-27T00:39:28.834325+0000","last_clean":"2024-05-27T00:39:28.834325+0000","last_became_active":"2024-05-27T00:39:21.585466+0000","last_became_peered":"2024-05-27T00:39:21.585466+0000","last_unstale":"2024-05-27T00:39:28.834325+0000","last_undegraded":"2024-05-27T00:39:28.834325+0000","last_fullsized":"2024-05-27T00:39:28.834325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.582020+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.581718+0000","last_became_peered":"2024-05-27T00:39:20.581718+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834411+0000","last_change":"2024-05-27T00:39:25.961052+0000","last_active":"2024-05-27T00:39:28.834411+0000","last_peered":"2024-05-27T00:39:28.834411+0000","last_clean":"2024-05-27T00:39:28.834411+0000","last_became_active":"2024-05-27T00:39:25.960632+0000","last_became_peered":"2024-05-27T00:39:25.960632+0000","last_unstale":"2024-05-27T00:39:28.834411+0000","last_undegraded":"2024-05-27T00:39:28.834411+0000","last_fullsized":"2024-05-27T00:39:28.834411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610612+0000","last_change":"2024-05-27T00:39:21.587149+0000","last_active":"2024-05-27T00:39:27.610612+0000","last_peered":"2024-05-27T00:39:27.610612+0000","last_clean":"2024-05-27T00:39:27.610612+0000","last_became_active":"2024-05-27T00:39:21.585877+0000","last_became_peered":"2024-05-27T00:39:21.585877+0000","last_unstale":"2024-05-27T00:39:27.610612+0000","last_undegraded":"2024-05-27T00:39:27.610612+0000","last_fullsized":"2024-05-27T00:39:27.610612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604682+0000","last_change":"2024-05-27T00:39:20.575827+0000","last_active":"2024-05-27T00:39:27.604682+0000","last_peered":"2024-05-27T00:39:27.604682+0000","last_clean":"2024-05-27T00:39:27.604682+0000","last_became_active":"2024-05-27T00:39:20.575555+0000","last_became_peered":"2024-05-27T00:39:20.575555+0000","last_unstale":"2024-05-27T00:39:27.604682+0000","last_undegraded":"2024-05-27T00:39:27.604682+0000","last_fullsized":"2024-05-27T00:39:27.604682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133662+0000","last_change":"2024-05-27T00:39:25.626022+0000","last_active":"2024-05-27T00:39:28.133662+0000","last_peered":"2024-05-27T00:39:28.133662+0000","last_clean":"2024-05-27T00:39:28.133662+0000","last_became_active":"2024-05-27T00:39:25.625643+0000","last_became_peered":"2024-05-27T00:39:25.625643+0000","last_unstale":"2024-05-27T00:39:28.133662+0000","last_undegraded":"2024-05-27T00:39:28.133662+0000","last_fullsized":"2024-05-27T00:39:28.133662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833753+0000","last_change":"2024-05-27T00:39:21.599339+0000","last_active":"2024-05-27T00:39:28.833753+0000","last_peered":"2024-05-27T00:39:28.833753+0000","last_clean":"2024-05-27T00:39:28.833753+0000","last_became_active":"2024-05-27T00:39:21.598994+0000","last_became_peered":"2024-05-27T00:39:21.598994+0000","last_unstale":"2024-05-27T00:39:28.833753+0000","last_undegraded":"2024-05-27T00:39:28.833753+0000","last_fullsized":"2024-05-27T00:39:28.833753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:13.702562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831770+0000","last_change":"2024-05-27T00:39:20.572771+0000","last_active":"2024-05-27T00:39:28.831770+0000","last_peered":"2024-05-27T00:39:28.831770+0000","last_clean":"2024-05-27T00:39:28.831770+0000","last_became_active":"2024-05-27T00:39:20.572486+0000","last_became_peered":"2024-05-27T00:39:20.572486+0000","last_unstale":"2024-05-27T00:39:28.831770+0000","last_undegraded":"2024-05-27T00:39:28.831770+0000","last_fullsized":"2024-05-27T00:39:28.831770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609790+0000","last_change":"2024-05-27T00:39:25.962926+0000","last_active":"2024-05-27T00:39:27.609790+0000","last_peered":"2024-05-27T00:39:27.609790+0000","last_clean":"2024-05-27T00:39:27.609790+0000","last_became_active":"2024-05-27T00:39:25.962471+0000","last_became_peered":"2024-05-27T00:39:25.962471+0000","last_unstale":"2024-05-27T00:39:27.609790+0000","last_undegraded":"2024-05-27T00:39:27.609790+0000","last_fullsized":"2024-05-27T00:39:27.609790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833146+0000","last_change":"2024-05-27T00:39:21.596571+0000","last_active":"2024-05-27T00:39:28.833146+0000","last_peered":"2024-05-27T00:39:28.833146+0000","last_clean":"2024-05-27T00:39:28.833146+0000","last_became_active":"2024-05-27T00:39:21.596278+0000","last_became_peered":"2024-05-27T00:39:21.596278+0000","last_unstale":"2024-05-27T00:39:28.833146+0000","last_undegraded":"2024-05-27T00:39:28.833146+0000","last_fullsized":"2024-05-27T00:39:28.833146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:42.881822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832599+0000","last_change":"2024-05-27T00:39:20.575461+0000","last_active":"2024-05-27T00:39:28.832599+0000","last_peered":"2024-05-27T00:39:28.832599+0000","last_clean":"2024-05-27T00:39:28.832599+0000","last_became_active":"2024-05-27T00:39:20.575168+0000","last_became_peered":"2024-05-27T00:39:20.575168+0000","last_unstale":"2024-05-27T00:39:28.832599+0000","last_undegraded":"2024-05-27T00:39:28.832599+0000","last_fullsized":"2024-05-27T00:39:28.832599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:04:51.312690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832581+0000","last_change":"2024-05-27T00:39:25.627556+0000","last_active":"2024-05-27T00:39:28.832581+0000","last_peered":"2024-05-27T00:39:28.832581+0000","last_clean":"2024-05-27T00:39:28.832581+0000","last_became_active":"2024-05-27T00:39:25.627169+0000","last_became_peered":"2024-05-27T00:39:25.627169+0000","last_unstale":"2024-05-27T00:39:28.832581+0000","last_undegraded":"2024-05-27T00:39:28.832581+0000","last_fullsized":"2024-05-27T00:39:28.832581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832699+0000","last_change":"2024-05-27T00:39:21.596905+0000","last_active":"2024-05-27T00:39:28.832699+0000","last_peered":"2024-05-27T00:39:28.832699+0000","last_clean":"2024-05-27T00:39:28.832699+0000","last_became_active":"2024-05-27T00:39:21.596586+0000","last_became_peered":"2024-05-27T00:39:21.596586+0000","last_unstale":"2024-05-27T00:39:28.832699+0000","last_undegraded":"2024-05-27T00:39:28.832699+0000","last_fullsized":"2024-05-27T00:39:28.832699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573588+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.573302+0000","last_became_peered":"2024-05-27T00:39:20.573302+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.620609+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.620185+0000","last_became_peered":"2024-05-27T00:39:25.620185+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835033+0000","last_change":"2024-05-27T00:39:21.593089+0000","last_active":"2024-05-27T00:39:28.835033+0000","last_peered":"2024-05-27T00:39:28.835033+0000","last_clean":"2024-05-27T00:39:28.835033+0000","last_became_active":"2024-05-27T00:39:21.592034+0000","last_became_peered":"2024-05-27T00:39:21.592034+0000","last_unstale":"2024-05-27T00:39:28.835033+0000","last_undegraded":"2024-05-27T00:39:28.835033+0000","last_fullsized":"2024-05-27T00:39:28.835033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834807+0000","last_change":"2024-05-27T00:39:20.576289+0000","last_active":"2024-05-27T00:39:28.834807+0000","last_peered":"2024-05-27T00:39:28.834807+0000","last_clean":"2024-05-27T00:39:28.834807+0000","last_became_active":"2024-05-27T00:39:20.576008+0000","last_became_peered":"2024-05-27T00:39:20.576008+0000","last_unstale":"2024-05-27T00:39:28.834807+0000","last_undegraded":"2024-05-27T00:39:28.834807+0000","last_fullsized":"2024-05-27T00:39:28.834807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610207+0000","last_change":"2024-05-27T00:39:25.964215+0000","last_active":"2024-05-27T00:39:27.610207+0000","last_peered":"2024-05-27T00:39:27.610207+0000","last_clean":"2024-05-27T00:39:27.610207+0000","last_became_active":"2024-05-27T00:39:25.963695+0000","last_became_peered":"2024-05-27T00:39:25.963695+0000","last_unstale":"2024-05-27T00:39:27.610207+0000","last_undegraded":"2024-05-27T00:39:27.610207+0000","last_fullsized":"2024-05-27T00:39:27.610207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:00:27.182513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833721+0000","last_change":"2024-05-27T00:39:21.599920+0000","last_active":"2024-05-27T00:39:28.833721+0000","last_peered":"2024-05-27T00:39:28.833721+0000","last_clean":"2024-05-27T00:39:28.833721+0000","last_became_active":"2024-05-27T00:39:21.597737+0000","last_became_peered":"2024-05-27T00:39:21.597737+0000","last_unstale":"2024-05-27T00:39:28.833721+0000","last_undegraded":"2024-05-27T00:39:28.833721+0000","last_fullsized":"2024-05-27T00:39:28.833721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:38.014686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610101+0000","last_change":"2024-05-27T00:39:20.571178+0000","last_active":"2024-05-27T00:39:27.610101+0000","last_peered":"2024-05-27T00:39:27.610101+0000","last_clean":"2024-05-27T00:39:27.610101+0000","last_became_active":"2024-05-27T00:39:20.570884+0000","last_became_peered":"2024-05-27T00:39:20.570884+0000","last_unstale":"2024-05-27T00:39:27.610101+0000","last_undegraded":"2024-05-27T00:39:27.610101+0000","last_fullsized":"2024-05-27T00:39:27.610101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610204+0000","last_change":"2024-05-27T00:39:25.961014+0000","last_active":"2024-05-27T00:39:27.610204+0000","last_peered":"2024-05-27T00:39:27.610204+0000","last_clean":"2024-05-27T00:39:27.610204+0000","last_became_active":"2024-05-27T00:39:25.960630+0000","last_became_peered":"2024-05-27T00:39:25.960630+0000","last_unstale":"2024-05-27T00:39:27.610204+0000","last_undegraded":"2024-05-27T00:39:27.610204+0000","last_fullsized":"2024-05-27T00:39:27.610204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:53:07.966829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133320+0000","last_change":"2024-05-27T00:39:21.583015+0000","last_active":"2024-05-27T00:39:28.133320+0000","last_peered":"2024-05-27T00:39:28.133320+0000","last_clean":"2024-05-27T00:39:28.133320+0000","last_became_active":"2024-05-27T00:39:21.582722+0000","last_became_peered":"2024-05-27T00:39:21.582722+0000","last_unstale":"2024-05-27T00:39:28.133320+0000","last_undegraded":"2024-05-27T00:39:28.133320+0000","last_fullsized":"2024-05-27T00:39:28.133320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:04.988100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133718+0000","last_change":"2024-05-27T00:39:20.575435+0000","last_active":"2024-05-27T00:39:28.133718+0000","last_peered":"2024-05-27T00:39:28.133718+0000","last_clean":"2024-05-27T00:39:28.133718+0000","last_became_active":"2024-05-27T00:39:20.575144+0000","last_became_peered":"2024-05-27T00:39:20.575144+0000","last_unstale":"2024-05-27T00:39:28.133718+0000","last_undegraded":"2024-05-27T00:39:28.133718+0000","last_fullsized":"2024-05-27T00:39:28.133718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:25:38.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610849+0000","last_change":"2024-05-27T00:39:25.961559+0000","last_active":"2024-05-27T00:39:27.610849+0000","last_peered":"2024-05-27T00:39:27.610849+0000","last_clean":"2024-05-27T00:39:27.610849+0000","last_became_active":"2024-05-27T00:39:25.961155+0000","last_became_peered":"2024-05-27T00:39:25.961155+0000","last_unstale":"2024-05-27T00:39:27.610849+0000","last_undegraded":"2024-05-27T00:39:27.610849+0000","last_fullsized":"2024-05-27T00:39:27.610849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834212+0000","last_change":"2024-05-27T00:39:21.593738+0000","last_active":"2024-05-27T00:39:28.834212+0000","last_peered":"2024-05-27T00:39:28.834212+0000","last_clean":"2024-05-27T00:39:28.834212+0000","last_became_active":"2024-05-27T00:39:21.591298+0000","last_became_peered":"2024-05-27T00:39:21.591298+0000","last_unstale":"2024-05-27T00:39:28.834212+0000","last_undegraded":"2024-05-27T00:39:28.834212+0000","last_fullsized":"2024-05-27T00:39:28.834212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:32.251423+0000","last_change":"2024-05-27T00:39:20.577937+0000","last_active":"2024-05-27T00:39:32.251423+0000","last_peered":"2024-05-27T00:39:32.251423+0000","last_clean":"2024-05-27T00:39:32.251423+0000","last_became_active":"2024-05-27T00:39:20.577630+0000","last_became_peered":"2024-05-27T00:39:20.577630+0000","last_unstale":"2024-05-27T00:39:32.251423+0000","last_undegraded":"2024-05-27T00:39:32.251423+0000","last_fullsized":"2024-05-27T00:39:32.251423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604388+0000","last_change":"2024-05-27T00:39:25.620344+0000","last_active":"2024-05-27T00:39:27.604388+0000","last_peered":"2024-05-27T00:39:27.604388+0000","last_clean":"2024-05-27T00:39:27.604388+0000","last_became_active":"2024-05-27T00:39:25.618090+0000","last_became_peered":"2024-05-27T00:39:25.618090+0000","last_unstale":"2024-05-27T00:39:27.604388+0000","last_undegraded":"2024-05-27T00:39:27.604388+0000","last_fullsized":"2024-05-27T00:39:27.604388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834698+0000","last_change":"2024-05-27T00:39:21.593265+0000","last_active":"2024-05-27T00:39:28.834698+0000","last_peered":"2024-05-27T00:39:28.834698+0000","last_clean":"2024-05-27T00:39:28.834698+0000","last_became_active":"2024-05-27T00:39:21.592984+0000","last_became_peered":"2024-05-27T00:39:21.592984+0000","last_unstale":"2024-05-27T00:39:28.834698+0000","last_undegraded":"2024-05-27T00:39:28.834698+0000","last_fullsized":"2024-05-27T00:39:28.834698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:06.451979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604736+0000","last_change":"2024-05-27T00:39:20.570519+0000","last_active":"2024-05-27T00:39:27.604736+0000","last_peered":"2024-05-27T00:39:27.604736+0000","last_clean":"2024-05-27T00:39:27.604736+0000","last_became_active":"2024-05-27T00:39:20.570230+0000","last_became_peered":"2024-05-27T00:39:20.570230+0000","last_unstale":"2024-05-27T00:39:27.604736+0000","last_undegraded":"2024-05-27T00:39:27.604736+0000","last_fullsized":"2024-05-27T00:39:27.604736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834196+0000","last_change":"2024-05-27T00:39:25.628554+0000","last_active":"2024-05-27T00:39:28.834196+0000","last_peered":"2024-05-27T00:39:28.834196+0000","last_clean":"2024-05-27T00:39:28.834196+0000","last_became_active":"2024-05-27T00:39:25.627971+0000","last_became_peered":"2024-05-27T00:39:25.627971+0000","last_unstale":"2024-05-27T00:39:28.834196+0000","last_undegraded":"2024-05-27T00:39:28.834196+0000","last_fullsized":"2024-05-27T00:39:28.834196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133800+0000","last_change":"2024-05-27T00:39:21.576653+0000","last_active":"2024-05-27T00:39:28.133800+0000","last_peered":"2024-05-27T00:39:28.133800+0000","last_clean":"2024-05-27T00:39:28.133800+0000","last_became_active":"2024-05-27T00:39:21.576383+0000","last_became_peered":"2024-05-27T00:39:21.576383+0000","last_unstale":"2024-05-27T00:39:28.133800+0000","last_undegraded":"2024-05-27T00:39:28.133800+0000","last_fullsized":"2024-05-27T00:39:28.133800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610702+0000","last_change":"2024-05-27T00:39:20.575867+0000","last_active":"2024-05-27T00:39:27.610702+0000","last_peered":"2024-05-27T00:39:27.610702+0000","last_clean":"2024-05-27T00:39:27.610702+0000","last_became_active":"2024-05-27T00:39:20.575557+0000","last_became_peered":"2024-05-27T00:39:20.575557+0000","last_unstale":"2024-05-27T00:39:27.610702+0000","last_undegraded":"2024-05-27T00:39:27.610702+0000","last_fullsized":"2024-05-27T00:39:27.610702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609795+0000","last_change":"2024-05-27T00:39:25.627020+0000","last_active":"2024-05-27T00:39:27.609795+0000","last_peered":"2024-05-27T00:39:27.609795+0000","last_clean":"2024-05-27T00:39:27.609795+0000","last_became_active":"2024-05-27T00:39:25.626629+0000","last_became_peered":"2024-05-27T00:39:25.626629+0000","last_unstale":"2024-05-27T00:39:27.609795+0000","last_undegraded":"2024-05-27T00:39:27.609795+0000","last_fullsized":"2024-05-27T00:39:27.609795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:57.470204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610617+0000","last_change":"2024-05-27T00:39:21.584614+0000","last_active":"2024-05-27T00:39:27.610617+0000","last_peered":"2024-05-27T00:39:27.610617+0000","last_clean":"2024-05-27T00:39:27.610617+0000","last_became_active":"2024-05-27T00:39:21.584349+0000","last_became_peered":"2024-05-27T00:39:21.584349+0000","last_unstale":"2024-05-27T00:39:27.610617+0000","last_undegraded":"2024-05-27T00:39:27.610617+0000","last_fullsized":"2024-05-27T00:39:27.610617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834142+0000","last_change":"2024-05-27T00:39:20.575901+0000","last_active":"2024-05-27T00:39:28.834142+0000","last_peered":"2024-05-27T00:39:28.834142+0000","last_clean":"2024-05-27T00:39:28.834142+0000","last_became_active":"2024-05-27T00:39:20.575521+0000","last_became_peered":"2024-05-27T00:39:20.575521+0000","last_unstale":"2024-05-27T00:39:28.834142+0000","last_undegraded":"2024-05-27T00:39:28.834142+0000","last_fullsized":"2024-05-27T00:39:28.834142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609706+0000","last_change":"2024-05-27T00:39:25.961740+0000","last_active":"2024-05-27T00:39:27.609706+0000","last_peered":"2024-05-27T00:39:27.609706+0000","last_clean":"2024-05-27T00:39:27.609706+0000","last_became_active":"2024-05-27T00:39:25.960524+0000","last_became_peered":"2024-05-27T00:39:25.960524+0000","last_unstale":"2024-05-27T00:39:27.609706+0000","last_undegraded":"2024-05-27T00:39:27.609706+0000","last_fullsized":"2024-05-27T00:39:27.609706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:38:40.065860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133581+0000","last_change":"2024-05-27T00:39:25.961849+0000","last_active":"2024-05-27T00:39:28.133581+0000","last_peered":"2024-05-27T00:39:28.133581+0000","last_clean":"2024-05-27T00:39:28.133581+0000","last_became_active":"2024-05-27T00:39:25.960998+0000","last_became_peered":"2024-05-27T00:39:25.960998+0000","last_unstale":"2024-05-27T00:39:28.133581+0000","last_undegraded":"2024-05-27T00:39:28.133581+0000","last_fullsized":"2024-05-27T00:39:28.133581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604056+0000","last_change":"2024-05-27T00:39:21.590501+0000","last_active":"2024-05-27T00:39:27.604056+0000","last_peered":"2024-05-27T00:39:27.604056+0000","last_clean":"2024-05-27T00:39:27.604056+0000","last_became_active":"2024-05-27T00:39:21.590246+0000","last_became_peered":"2024-05-27T00:39:21.590246+0000","last_unstale":"2024-05-27T00:39:27.604056+0000","last_undegraded":"2024-05-27T00:39:27.604056+0000","last_fullsized":"2024-05-27T00:39:27.604056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609915+0000","last_change":"2024-05-27T00:39:20.574994+0000","last_active":"2024-05-27T00:39:27.609915+0000","last_peered":"2024-05-27T00:39:27.609915+0000","last_clean":"2024-05-27T00:39:27.609915+0000","last_became_active":"2024-05-27T00:39:20.574713+0000","last_became_peered":"2024-05-27T00:39:20.574713+0000","last_unstale":"2024-05-27T00:39:27.609915+0000","last_undegraded":"2024-05-27T00:39:27.609915+0000","last_fullsized":"2024-05-27T00:39:27.609915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835197+0000","last_change":"2024-05-27T00:39:25.629723+0000","last_active":"2024-05-27T00:39:28.835197+0000","last_peered":"2024-05-27T00:39:28.835197+0000","last_clean":"2024-05-27T00:39:28.835197+0000","last_became_active":"2024-05-27T00:39:25.628168+0000","last_became_peered":"2024-05-27T00:39:25.628168+0000","last_unstale":"2024-05-27T00:39:28.835197+0000","last_undegraded":"2024-05-27T00:39:28.835197+0000","last_fullsized":"2024-05-27T00:39:28.835197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609695+0000","last_change":"2024-05-27T00:39:21.591614+0000","last_active":"2024-05-27T00:39:27.609695+0000","last_peered":"2024-05-27T00:39:27.609695+0000","last_clean":"2024-05-27T00:39:27.609695+0000","last_became_active":"2024-05-27T00:39:21.591359+0000","last_became_peered":"2024-05-27T00:39:21.591359+0000","last_unstale":"2024-05-27T00:39:27.609695+0000","last_undegraded":"2024-05-27T00:39:27.609695+0000","last_fullsized":"2024-05-27T00:39:27.609695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610097+0000","last_change":"2024-05-27T00:39:20.575532+0000","last_active":"2024-05-27T00:39:27.610097+0000","last_peered":"2024-05-27T00:39:27.610097+0000","last_clean":"2024-05-27T00:39:27.610097+0000","last_became_active":"2024-05-27T00:39:20.575201+0000","last_became_peered":"2024-05-27T00:39:20.575201+0000","last_unstale":"2024-05-27T00:39:27.610097+0000","last_undegraded":"2024-05-27T00:39:27.610097+0000","last_fullsized":"2024-05-27T00:39:27.610097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834640+0000","last_change":"2024-05-27T00:39:20.577031+0000","last_active":"2024-05-27T00:39:28.834640+0000","last_peered":"2024-05-27T00:39:28.834640+0000","last_clean":"2024-05-27T00:39:28.834640+0000","last_became_active":"2024-05-27T00:39:20.576619+0000","last_became_peered":"2024-05-27T00:39:20.576619+0000","last_unstale":"2024-05-27T00:39:28.834640+0000","last_undegraded":"2024-05-27T00:39:28.834640+0000","last_fullsized":"2024-05-27T00:39:28.834640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133684+0000","last_change":"2024-05-27T00:39:21.580671+0000","last_active":"2024-05-27T00:39:28.133684+0000","last_peered":"2024-05-27T00:39:28.133684+0000","last_clean":"2024-05-27T00:39:28.133684+0000","last_became_active":"2024-05-27T00:39:21.580434+0000","last_became_peered":"2024-05-27T00:39:21.580434+0000","last_unstale":"2024-05-27T00:39:28.133684+0000","last_undegraded":"2024-05-27T00:39:28.133684+0000","last_fullsized":"2024-05-27T00:39:28.133684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832153+0000","last_change":"2024-05-27T00:39:25.624658+0000","last_active":"2024-05-27T00:39:28.832153+0000","last_peered":"2024-05-27T00:39:28.832153+0000","last_clean":"2024-05-27T00:39:28.832153+0000","last_became_active":"2024-05-27T00:39:25.624289+0000","last_became_peered":"2024-05-27T00:39:25.624289+0000","last_unstale":"2024-05-27T00:39:28.832153+0000","last_undegraded":"2024-05-27T00:39:28.832153+0000","last_fullsized":"2024-05-27T00:39:28.832153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:22.409161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609979+0000","last_change":"2024-05-27T00:39:20.572852+0000","last_active":"2024-05-27T00:39:27.609979+0000","last_peered":"2024-05-27T00:39:27.609979+0000","last_clean":"2024-05-27T00:39:27.609979+0000","last_became_active":"2024-05-27T00:39:20.572562+0000","last_became_peered":"2024-05-27T00:39:20.572562+0000","last_unstale":"2024-05-27T00:39:27.609979+0000","last_undegraded":"2024-05-27T00:39:27.609979+0000","last_fullsized":"2024-05-27T00:39:27.609979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833202+0000","last_change":"2024-05-27T00:39:21.587846+0000","last_active":"2024-05-27T00:39:28.833202+0000","last_peered":"2024-05-27T00:39:28.833202+0000","last_clean":"2024-05-27T00:39:28.833202+0000","last_became_active":"2024-05-27T00:39:21.587579+0000","last_became_peered":"2024-05-27T00:39:21.587579+0000","last_unstale":"2024-05-27T00:39:28.833202+0000","last_undegraded":"2024-05-27T00:39:28.833202+0000","last_fullsized":"2024-05-27T00:39:28.833202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:47.382970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610160+0000","last_change":"2024-05-27T00:39:25.628905+0000","last_active":"2024-05-27T00:39:27.610160+0000","last_peered":"2024-05-27T00:39:27.610160+0000","last_clean":"2024-05-27T00:39:27.610160+0000","last_became_active":"2024-05-27T00:39:25.628229+0000","last_became_peered":"2024-05-27T00:39:25.628229+0000","last_unstale":"2024-05-27T00:39:27.610160+0000","last_undegraded":"2024-05-27T00:39:27.610160+0000","last_fullsized":"2024-05-27T00:39:27.610160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:40:24.502057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834905+0000","last_change":"2024-05-27T00:39:21.593542+0000","last_active":"2024-05-27T00:39:28.834905+0000","last_peered":"2024-05-27T00:39:28.834905+0000","last_clean":"2024-05-27T00:39:28.834905+0000","last_became_active":"2024-05-27T00:39:21.592466+0000","last_became_peered":"2024-05-27T00:39:21.592466+0000","last_unstale":"2024-05-27T00:39:28.834905+0000","last_undegraded":"2024-05-27T00:39:28.834905+0000","last_fullsized":"2024-05-27T00:39:28.834905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831986+0000","last_change":"2024-05-27T00:39:21.586207+0000","last_active":"2024-05-27T00:39:28.831986+0000","last_peered":"2024-05-27T00:39:28.831986+0000","last_clean":"2024-05-27T00:39:28.831986+0000","last_became_active":"2024-05-27T00:39:21.585899+0000","last_became_peered":"2024-05-27T00:39:21.585899+0000","last_unstale":"2024-05-27T00:39:28.831986+0000","last_undegraded":"2024-05-27T00:39:28.831986+0000","last_fullsized":"2024-05-27T00:39:28.831986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610077+0000","last_change":"2024-05-27T00:39:21.589304+0000","last_active":"2024-05-27T00:39:27.610077+0000","last_peered":"2024-05-27T00:39:27.610077+0000","last_clean":"2024-05-27T00:39:27.610077+0000","last_became_active":"2024-05-27T00:39:21.588424+0000","last_became_peered":"2024-05-27T00:39:21.588424+0000","last_unstale":"2024-05-27T00:39:27.610077+0000","last_undegraded":"2024-05-27T00:39:27.610077+0000","last_fullsized":"2024-05-27T00:39:27.610077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133594+0000","last_change":"2024-05-27T00:39:21.587527+0000","last_active":"2024-05-27T00:39:28.133594+0000","last_peered":"2024-05-27T00:39:28.133594+0000","last_clean":"2024-05-27T00:39:28.133594+0000","last_became_active":"2024-05-27T00:39:21.587271+0000","last_became_peered":"2024-05-27T00:39:21.587271+0000","last_unstale":"2024-05-27T00:39:28.133594+0000","last_undegraded":"2024-05-27T00:39:28.133594+0000","last_fullsized":"2024-05-27T00:39:28.133594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610312+0000","last_change":"2024-05-27T00:39:21.600567+0000","last_active":"2024-05-27T00:39:27.610312+0000","last_peered":"2024-05-27T00:39:27.610312+0000","last_clean":"2024-05-27T00:39:27.610312+0000","last_became_active":"2024-05-27T00:39:21.600281+0000","last_became_peered":"2024-05-27T00:39:21.600281+0000","last_unstale":"2024-05-27T00:39:27.610312+0000","last_undegraded":"2024-05-27T00:39:27.610312+0000","last_fullsized":"2024-05-27T00:39:27.610312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604571+0000","last_change":"2024-05-27T00:39:21.591200+0000","last_active":"2024-05-27T00:39:27.604571+0000","last_peered":"2024-05-27T00:39:27.604571+0000","last_clean":"2024-05-27T00:39:27.604571+0000","last_became_active":"2024-05-27T00:39:21.590918+0000","last_became_peered":"2024-05-27T00:39:21.590918+0000","last_unstale":"2024-05-27T00:39:27.604571+0000","last_undegraded":"2024-05-27T00:39:27.604571+0000","last_fullsized":"2024-05-27T00:39:27.604571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835139+0000","last_change":"2024-05-27T00:39:21.593373+0000","last_active":"2024-05-27T00:39:28.835139+0000","last_peered":"2024-05-27T00:39:28.835139+0000","last_clean":"2024-05-27T00:39:28.835139+0000","last_became_active":"2024-05-27T00:39:21.590646+0000","last_became_peered":"2024-05-27T00:39:21.590646+0000","last_unstale":"2024-05-27T00:39:28.835139+0000","last_undegraded":"2024-05-27T00:39:28.835139+0000","last_fullsized":"2024-05-27T00:39:28.835139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610260+0000","last_change":"2024-05-27T00:39:21.600531+0000","last_active":"2024-05-27T00:39:27.610260+0000","last_peered":"2024-05-27T00:39:27.610260+0000","last_clean":"2024-05-27T00:39:27.610260+0000","last_became_active":"2024-05-27T00:39:21.600236+0000","last_became_peered":"2024-05-27T00:39:21.600236+0000","last_unstale":"2024-05-27T00:39:27.610260+0000","last_undegraded":"2024-05-27T00:39:27.610260+0000","last_fullsized":"2024-05-27T00:39:27.610260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609848+0000","last_change":"2024-05-27T00:39:21.588532+0000","last_active":"2024-05-27T00:39:27.609848+0000","last_peered":"2024-05-27T00:39:27.609848+0000","last_clean":"2024-05-27T00:39:27.609848+0000","last_became_active":"2024-05-27T00:39:21.587475+0000","last_became_peered":"2024-05-27T00:39:21.587475+0000","last_unstale":"2024-05-27T00:39:27.609848+0000","last_undegraded":"2024-05-27T00:39:27.609848+0000","last_fullsized":"2024-05-27T00:39:27.609848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609903+0000","last_change":"2024-05-27T00:39:21.601094+0000","last_active":"2024-05-27T00:39:27.609903+0000","last_peered":"2024-05-27T00:39:27.609903+0000","last_clean":"2024-05-27T00:39:27.609903+0000","last_became_active":"2024-05-27T00:39:21.600345+0000","last_became_peered":"2024-05-27T00:39:21.600345+0000","last_unstale":"2024-05-27T00:39:27.609903+0000","last_undegraded":"2024-05-27T00:39:27.609903+0000","last_fullsized":"2024-05-27T00:39:27.609903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609931+0000","last_change":"2024-05-27T00:39:21.583984+0000","last_active":"2024-05-27T00:39:27.609931+0000","last_peered":"2024-05-27T00:39:27.609931+0000","last_clean":"2024-05-27T00:39:27.609931+0000","last_became_active":"2024-05-27T00:39:21.583601+0000","last_became_peered":"2024-05-27T00:39:21.583601+0000","last_unstale":"2024-05-27T00:39:27.609931+0000","last_undegraded":"2024-05-27T00:39:27.609931+0000","last_fullsized":"2024-05-27T00:39:27.609931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133437+0000","last_change":"2024-05-27T00:39:21.583713+0000","last_active":"2024-05-27T00:39:28.133437+0000","last_peered":"2024-05-27T00:39:28.133437+0000","last_clean":"2024-05-27T00:39:28.133437+0000","last_became_active":"2024-05-27T00:39:21.583367+0000","last_became_peered":"2024-05-27T00:39:21.583367+0000","last_unstale":"2024-05-27T00:39:28.133437+0000","last_undegraded":"2024-05-27T00:39:28.133437+0000","last_fullsized":"2024-05-27T00:39:28.133437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834031+0000","last_change":"2024-05-27T00:39:21.597145+0000","last_active":"2024-05-27T00:39:28.834031+0000","last_peered":"2024-05-27T00:39:28.834031+0000","last_clean":"2024-05-27T00:39:28.834031+0000","last_became_active":"2024-05-27T00:39:21.596737+0000","last_became_peered":"2024-05-27T00:39:21.596737+0000","last_unstale":"2024-05-27T00:39:28.834031+0000","last_undegraded":"2024-05-27T00:39:28.834031+0000","last_fullsized":"2024-05-27T00:39:28.834031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834671+0000","last_change":"2024-05-27T00:39:21.586729+0000","last_active":"2024-05-27T00:39:28.834671+0000","last_peered":"2024-05-27T00:39:28.834671+0000","last_clean":"2024-05-27T00:39:28.834671+0000","last_became_active":"2024-05-27T00:39:21.586428+0000","last_became_peered":"2024-05-27T00:39:21.586428+0000","last_unstale":"2024-05-27T00:39:28.834671+0000","last_undegraded":"2024-05-27T00:39:28.834671+0000","last_fullsized":"2024-05-27T00:39:28.834671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832018+0000","last_change":"2024-05-27T00:39:21.591955+0000","last_active":"2024-05-27T00:39:28.832018+0000","last_peered":"2024-05-27T00:39:28.832018+0000","last_clean":"2024-05-27T00:39:28.832018+0000","last_became_active":"2024-05-27T00:39:21.591645+0000","last_became_peered":"2024-05-27T00:39:21.591645+0000","last_unstale":"2024-05-27T00:39:28.832018+0000","last_undegraded":"2024-05-27T00:39:28.832018+0000","last_fullsized":"2024-05-27T00:39:28.832018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610938+0000","last_change":"2024-05-27T00:39:21.582309+0000","last_active":"2024-05-27T00:39:27.610938+0000","last_peered":"2024-05-27T00:39:27.610938+0000","last_clean":"2024-05-27T00:39:27.610938+0000","last_became_active":"2024-05-27T00:39:21.582029+0000","last_became_peered":"2024-05-27T00:39:21.582029+0000","last_unstale":"2024-05-27T00:39:27.610938+0000","last_undegraded":"2024-05-27T00:39:27.610938+0000","last_fullsized":"2024-05-27T00:39:27.610938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609792+0000","last_change":"2024-05-27T00:39:21.583269+0000","last_active":"2024-05-27T00:39:27.609792+0000","last_peered":"2024-05-27T00:39:27.609792+0000","last_clean":"2024-05-27T00:39:27.609792+0000","last_became_active":"2024-05-27T00:39:21.582827+0000","last_became_peered":"2024-05-27T00:39:21.582827+0000","last_unstale":"2024-05-27T00:39:27.609792+0000","last_undegraded":"2024-05-27T00:39:27.609792+0000","last_fullsized":"2024-05-27T00:39:27.609792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604091+0000","last_change":"2024-05-27T00:39:21.600540+0000","last_active":"2024-05-27T00:39:27.604091+0000","last_peered":"2024-05-27T00:39:27.604091+0000","last_clean":"2024-05-27T00:39:27.604091+0000","last_became_active":"2024-05-27T00:39:21.600270+0000","last_became_peered":"2024-05-27T00:39:21.600270+0000","last_unstale":"2024-05-27T00:39:27.604091+0000","last_undegraded":"2024-05-27T00:39:27.604091+0000","last_fullsized":"2024-05-27T00:39:27.604091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133315+0000","last_change":"2024-05-27T00:39:21.587064+0000","last_active":"2024-05-27T00:39:28.133315+0000","last_peered":"2024-05-27T00:39:28.133315+0000","last_clean":"2024-05-27T00:39:28.133315+0000","last_became_active":"2024-05-27T00:39:21.586759+0000","last_became_peered":"2024-05-27T00:39:21.586759+0000","last_unstale":"2024-05-27T00:39:28.133315+0000","last_undegraded":"2024-05-27T00:39:28.133315+0000","last_fullsized":"2024-05-27T00:39:28.133315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:18.374156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609806+0000","last_change":"2024-05-27T00:39:21.594918+0000","last_active":"2024-05-27T00:39:27.609806+0000","last_peered":"2024-05-27T00:39:27.609806+0000","last_clean":"2024-05-27T00:39:27.609806+0000","last_became_active":"2024-05-27T00:39:21.594670+0000","last_became_peered":"2024-05-27T00:39:21.594670+0000","last_unstale":"2024-05-27T00:39:27.609806+0000","last_undegraded":"2024-05-27T00:39:27.609806+0000","last_fullsized":"2024-05-27T00:39:27.609806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834590+0000","last_change":"2024-05-27T00:39:21.600525+0000","last_active":"2024-05-27T00:39:28.834590+0000","last_peered":"2024-05-27T00:39:28.834590+0000","last_clean":"2024-05-27T00:39:28.834590+0000","last_became_active":"2024-05-27T00:39:21.600012+0000","last_became_peered":"2024-05-27T00:39:21.600012+0000","last_unstale":"2024-05-27T00:39:28.834590+0000","last_undegraded":"2024-05-27T00:39:28.834590+0000","last_fullsized":"2024-05-27T00:39:28.834590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834173+0000","last_change":"2024-05-27T00:39:21.596158+0000","last_active":"2024-05-27T00:39:28.834173+0000","last_peered":"2024-05-27T00:39:28.834173+0000","last_clean":"2024-05-27T00:39:28.834173+0000","last_became_active":"2024-05-27T00:39:21.595655+0000","last_became_peered":"2024-05-27T00:39:21.595655+0000","last_unstale":"2024-05-27T00:39:28.834173+0000","last_undegraded":"2024-05-27T00:39:28.834173+0000","last_fullsized":"2024-05-27T00:39:28.834173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835071+0000","last_change":"2024-05-27T00:39:21.594957+0000","last_active":"2024-05-27T00:39:28.835071+0000","last_peered":"2024-05-27T00:39:28.835071+0000","last_clean":"2024-05-27T00:39:28.835071+0000","last_became_active":"2024-05-27T00:39:21.594682+0000","last_became_peered":"2024-05-27T00:39:21.594682+0000","last_unstale":"2024-05-27T00:39:28.835071+0000","last_undegraded":"2024-05-27T00:39:28.835071+0000","last_fullsized":"2024-05-27T00:39:28.835071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833653+0000","last_change":"2024-05-27T00:39:21.589852+0000","last_active":"2024-05-27T00:39:28.833653+0000","last_peered":"2024-05-27T00:39:28.833653+0000","last_clean":"2024-05-27T00:39:28.833653+0000","last_became_active":"2024-05-27T00:39:21.589514+0000","last_became_peered":"2024-05-27T00:39:21.589514+0000","last_unstale":"2024-05-27T00:39:28.833653+0000","last_undegraded":"2024-05-27T00:39:28.833653+0000","last_fullsized":"2024-05-27T00:39:28.833653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:08.796084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609583+0000","last_change":"2024-05-27T00:39:21.586007+0000","last_active":"2024-05-27T00:39:27.609583+0000","last_peered":"2024-05-27T00:39:27.609583+0000","last_clean":"2024-05-27T00:39:27.609583+0000","last_became_active":"2024-05-27T00:39:21.585708+0000","last_became_peered":"2024-05-27T00:39:21.585708+0000","last_unstale":"2024-05-27T00:39:27.609583+0000","last_undegraded":"2024-05-27T00:39:27.609583+0000","last_fullsized":"2024-05-27T00:39:27.609583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832658+0000","last_change":"2024-05-27T00:39:21.596377+0000","last_active":"2024-05-27T00:39:28.832658+0000","last_peered":"2024-05-27T00:39:28.832658+0000","last_clean":"2024-05-27T00:39:28.832658+0000","last_became_active":"2024-05-27T00:39:21.596055+0000","last_became_peered":"2024-05-27T00:39:21.596055+0000","last_unstale":"2024-05-27T00:39:28.832658+0000","last_undegraded":"2024-05-27T00:39:28.832658+0000","last_fullsized":"2024-05-27T00:39:28.832658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609929+0000","last_change":"2024-05-27T00:39:21.583846+0000","last_active":"2024-05-27T00:39:27.609929+0000","last_peered":"2024-05-27T00:39:27.609929+0000","last_clean":"2024-05-27T00:39:27.609929+0000","last_became_active":"2024-05-27T00:39:21.583284+0000","last_became_peered":"2024-05-27T00:39:21.583284+0000","last_unstale":"2024-05-27T00:39:27.609929+0000","last_undegraded":"2024-05-27T00:39:27.609929+0000","last_fullsized":"2024-05-27T00:39:27.609929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833536+0000","last_change":"2024-05-27T00:39:21.586873+0000","last_active":"2024-05-27T00:39:28.833536+0000","last_peered":"2024-05-27T00:39:28.833536+0000","last_clean":"2024-05-27T00:39:28.833536+0000","last_became_active":"2024-05-27T00:39:21.586593+0000","last_became_peered":"2024-05-27T00:39:21.586593+0000","last_unstale":"2024-05-27T00:39:28.833536+0000","last_undegraded":"2024-05-27T00:39:28.833536+0000","last_fullsized":"2024-05-27T00:39:28.833536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610399+0000","last_change":"2024-05-27T00:39:21.592963+0000","last_active":"2024-05-27T00:39:27.610399+0000","last_peered":"2024-05-27T00:39:27.610399+0000","last_clean":"2024-05-27T00:39:27.610399+0000","last_became_active":"2024-05-27T00:39:21.588049+0000","last_became_peered":"2024-05-27T00:39:21.588049+0000","last_unstale":"2024-05-27T00:39:27.610399+0000","last_undegraded":"2024-05-27T00:39:27.610399+0000","last_fullsized":"2024-05-27T00:39:27.610399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133490+0000","last_change":"2024-05-27T00:39:21.588239+0000","last_active":"2024-05-27T00:39:28.133490+0000","last_peered":"2024-05-27T00:39:28.133490+0000","last_clean":"2024-05-27T00:39:28.133490+0000","last_became_active":"2024-05-27T00:39:21.587936+0000","last_became_peered":"2024-05-27T00:39:21.587936+0000","last_unstale":"2024-05-27T00:39:28.133490+0000","last_undegraded":"2024-05-27T00:39:28.133490+0000","last_fullsized":"2024-05-27T00:39:28.133490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604778+0000","last_change":"2024-05-27T00:39:21.591553+0000","last_active":"2024-05-27T00:39:27.604778+0000","last_peered":"2024-05-27T00:39:27.604778+0000","last_clean":"2024-05-27T00:39:27.604778+0000","last_became_active":"2024-05-27T00:39:21.591286+0000","last_became_peered":"2024-05-27T00:39:21.591286+0000","last_unstale":"2024-05-27T00:39:27.604778+0000","last_undegraded":"2024-05-27T00:39:27.604778+0000","last_fullsized":"2024-05-27T00:39:27.604778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833459+0000","last_change":"2024-05-27T00:39:21.591695+0000","last_active":"2024-05-27T00:39:28.833459+0000","last_peered":"2024-05-27T00:39:28.833459+0000","last_clean":"2024-05-27T00:39:28.833459+0000","last_became_active":"2024-05-27T00:39:21.591384+0000","last_became_peered":"2024-05-27T00:39:21.591384+0000","last_unstale":"2024-05-27T00:39:28.833459+0000","last_undegraded":"2024-05-27T00:39:28.833459+0000","last_fullsized":"2024-05-27T00:39:28.833459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:39:45.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:39:45.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:39:45.702 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:39:45.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 Processor -- start 2024-05-27T00:39:45.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 -- start start 2024-05-27T00:39:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33f0161e50 0x7f33f01622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f33f00d2c10 0x7f33f00d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f006cc70 con 0x7f33f00b3630 2024-05-27T00:39:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f33f00d35c0 con 0x7f33f0161e50 2024-05-27T00:39:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.757+0000 7f33f5227640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f33f00d1e50 con 0x7f33f00d2c10 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33f0161e50 0x7f33f01622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f33f0161e50 0x7f33f01622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51262/0 (socket says 172.21.15.111:51262) 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 -- 172.21.15.111:0/984176074 learned_addr learned my addr 172.21.15.111:0/984176074 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33ef59e640 1 --2- 172.21.15.111:0/984176074 >> v2:172.21.15.153:3300/0 conn(0x7f33f00d2c10 0x7f33f00d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 -- 172.21.15.111:0/984176074 >> v2:172.21.15.153:3300/0 conn(0x7f33f00d2c10 msgr2=0x7f33f00d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 --2- 172.21.15.111:0/984176074 >> v2:172.21.15.153:3300/0 conn(0x7f33f00d2c10 0x7f33f00d3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 -- 172.21.15.111:0/984176074 >> v1:172.21.15.153:6789/0 conn(0x7f33f00b3630 legacy=0x7f33f00b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.758+0000 7f33e7fff640 1 -- 172.21.15.111:0/984176074 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00d2130 con 0x7f33f0161e50 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33e7fff640 1 --2- 172.21.15.111:0/984176074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f01622e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33f008d480 tx=0x7f33d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45bc158f6792482 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33ee59c640 1 -- 172.21.15.111:0/984176074 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8037070 con 0x7f33f0161e50 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33ee59c640 1 -- 172.21.15.111:0/984176074 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d8032440 con 0x7f33f0161e50 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33ee59c640 1 -- 172.21.15.111:0/984176074 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8032810 con 0x7f33f0161e50 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 -- 172.21.15.111:0/984176074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 msgr2=0x7f33f01622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 --2- 172.21.15.111:0/984176074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f01622e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33f008d480 tx=0x7f33d802a230 comp rx=0 tx=0).stop 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 -- 172.21.15.111:0/984176074 shutdown_connections 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 --2- 172.21.15.111:0/984176074 >> v2:172.21.15.153:3300/0 conn(0x7f33f00d2c10 0x7f33f00d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 --2- 172.21.15.111:0/984176074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f01622e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 -- 172.21.15.111:0/984176074 >> 172.21.15.111:0/984176074 conn(0x7f33f00d42c0 msgr2=0x7f33f00d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 -- 172.21.15.111:0/984176074 shutdown_connections 2024-05-27T00:39:45.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.759+0000 7f33f5227640 1 -- 172.21.15.111:0/984176074 wait complete. 2024-05-27T00:39:45.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 Processor -- start 2024-05-27T00:39:45.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 -- start start 2024-05-27T00:39:45.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f00b3630 0x7f33f00cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.765 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d2c10 0x7f33f00aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f00cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00b79b0 con 0x7f33f0161e50 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00d26f0 con 0x7f33f00d2c10 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33f5227640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f33f006bfb0 con 0x7f33f00b3630 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.761+0000 7f33ef59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f00cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f00cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51270/0 (socket says 172.21.15.111:51270) 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 -- 172.21.15.111:0/386019475 learned_addr learned my addr 172.21.15.111:0/386019475 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33eed9d640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f00b3630 0x7f33f00cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33e7fff640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d2c10 0x7f33f00aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 -- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f00b3630 msgr2=0x7f33f00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f00b3630 0x7f33f00cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 -- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d2c10 msgr2=0x7f33f00aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d2c10 0x7f33f00aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 -- 172.21.15.111:0/386019475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00ce120 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33e7fff640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d2c10 0x7f33f00aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33ef59e640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f00cf360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33e800cd70 tx=0x7f33e800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8003b90 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e8003d30 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f00aafa0 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8004240 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.762+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f00ab560 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.763+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f00754c0 con 0x7f33f0161e50 2024-05-27T00:39:45.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.763+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33e80043e0 con 0x7f33f0161e50 2024-05-27T00:39:45.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.764+0000 7f33e77fe640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f33d006be30 0x7f33d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:45.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.764+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f33e808bca0 con 0x7f33f0161e50 2024-05-27T00:39:45.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.764+0000 7f33eed9d640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f33d006be30 0x7f33d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:45.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.765+0000 7f33eed9d640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f33d006be30 0x7f33d006e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004ea0 tx=0x7f33e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:45.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.768+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f33f00754c0 con 0x7f33f0161e50 2024-05-27T00:39:45.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.937+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33f0068680 con 0x7f33d006be30 2024-05-27T00:39:45.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.946+0000 7f33e77fe640 1 -- 172.21.15.111:0/386019475 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f33f0068680 con 0x7f33d006be30 2024-05-27T00:39:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:45.952 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.949+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f33d006be30 msgr2=0x7f33d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.949+0000 7f33f5227640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f33d006be30 0x7f33d006e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004ea0 tx=0x7f33e0009040 comp rx=0 tx=0).stop 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.949+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 msgr2=0x7f33f00cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.949+0000 7f33f5227640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f00cf360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33e800cd70 tx=0x7f33e800a660 comp rx=0 tx=0).stop 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 shutdown_connections 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f33d006be30 0x7f33d006e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33f0161e50 0x7f33f00cf360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33f00d2c10 0x7f33f00aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 --2- 172.21.15.111:0/386019475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f33f00b3630 0x7f33f00cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:45.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 >> 172.21.15.111:0/386019475 conn(0x7f33f00d42c0 msgr2=0x7f33f00b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:45.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 shutdown_connections 2024-05-27T00:39:45.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:45.950+0000 7f33f5227640 1 -- 172.21.15.111:0/386019475 wait complete. 2024-05-27T00:39:45.962 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-27T00:39:44.930364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003519"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133844+0000","last_change":"2024-05-27T00:39:20.574932+0000","last_active":"2024-05-27T00:39:28.133844+0000","last_peered":"2024-05-27T00:39:28.133844+0000","last_clean":"2024-05-27T00:39:28.133844+0000","last_became_active":"2024-05-27T00:39:20.574617+0000","last_became_peered":"2024-05-27T00:39:20.574617+0000","last_unstale":"2024-05-27T00:39:28.133844+0000","last_undegraded":"2024-05-27T00:39:28.133844+0000","last_fullsized":"2024-05-27T00:39:28.133844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610219+0000","last_change":"2024-05-27T00:39:21.594550+0000","last_active":"2024-05-27T00:39:27.610219+0000","last_peered":"2024-05-27T00:39:27.610219+0000","last_clean":"2024-05-27T00:39:27.610219+0000","last_became_active":"2024-05-27T00:39:21.594296+0000","last_became_peered":"2024-05-27T00:39:21.594296+0000","last_unstale":"2024-05-27T00:39:27.610219+0000","last_undegraded":"2024-05-27T00:39:27.610219+0000","last_fullsized":"2024-05-27T00:39:27.610219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609737+0000","last_change":"2024-05-27T00:39:25.965194+0000","last_active":"2024-05-27T00:39:27.609737+0000","last_peered":"2024-05-27T00:39:27.609737+0000","last_clean":"2024-05-27T00:39:27.609737+0000","last_became_active":"2024-05-27T00:39:25.964829+0000","last_became_peered":"2024-05-27T00:39:25.964829+0000","last_unstale":"2024-05-27T00:39:27.609737+0000","last_undegraded":"2024-05-27T00:39:27.609737+0000","last_fullsized":"2024-05-27T00:39:27.609737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:38.727603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610308+0000","last_change":"2024-05-27T00:39:20.569739+0000","last_active":"2024-05-27T00:39:27.610308+0000","last_peered":"2024-05-27T00:39:27.610308+0000","last_clean":"2024-05-27T00:39:27.610308+0000","last_became_active":"2024-05-27T00:39:20.569481+0000","last_became_peered":"2024-05-27T00:39:20.569481+0000","last_unstale":"2024-05-27T00:39:27.610308+0000","last_undegraded":"2024-05-27T00:39:27.610308+0000","last_fullsized":"2024-05-27T00:39:27.610308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:43.678819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133753+0000","last_change":"2024-05-27T00:39:21.576759+0000","last_active":"2024-05-27T00:39:28.133753+0000","last_peered":"2024-05-27T00:39:28.133753+0000","last_clean":"2024-05-27T00:39:28.133753+0000","last_became_active":"2024-05-27T00:39:21.576492+0000","last_became_peered":"2024-05-27T00:39:21.576492+0000","last_unstale":"2024-05-27T00:39:28.133753+0000","last_undegraded":"2024-05-27T00:39:28.133753+0000","last_fullsized":"2024-05-27T00:39:28.133753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833532+0000","last_change":"2024-05-27T00:39:25.626523+0000","last_active":"2024-05-27T00:39:28.833532+0000","last_peered":"2024-05-27T00:39:28.833532+0000","last_clean":"2024-05-27T00:39:28.833532+0000","last_became_active":"2024-05-27T00:39:25.626132+0000","last_became_peered":"2024-05-27T00:39:25.626132+0000","last_unstale":"2024-05-27T00:39:28.833532+0000","last_undegraded":"2024-05-27T00:39:28.833532+0000","last_fullsized":"2024-05-27T00:39:28.833532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834538+0000","last_change":"2024-05-27T00:39:20.577889+0000","last_active":"2024-05-27T00:39:28.834538+0000","last_peered":"2024-05-27T00:39:28.834538+0000","last_clean":"2024-05-27T00:39:28.834538+0000","last_became_active":"2024-05-27T00:39:20.577565+0000","last_became_peered":"2024-05-27T00:39:20.577565+0000","last_unstale":"2024-05-27T00:39:28.834538+0000","last_undegraded":"2024-05-27T00:39:28.834538+0000","last_fullsized":"2024-05-27T00:39:28.834538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:12:22.836110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610522+0000","last_change":"2024-05-27T00:39:21.580044+0000","last_active":"2024-05-27T00:39:27.610522+0000","last_peered":"2024-05-27T00:39:27.610522+0000","last_clean":"2024-05-27T00:39:27.610522+0000","last_became_active":"2024-05-27T00:39:21.579770+0000","last_became_peered":"2024-05-27T00:39:21.579770+0000","last_unstale":"2024-05-27T00:39:27.610522+0000","last_undegraded":"2024-05-27T00:39:27.610522+0000","last_fullsized":"2024-05-27T00:39:27.610522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:01.058661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833971+0000","last_change":"2024-05-27T00:39:25.627365+0000","last_active":"2024-05-27T00:39:28.833971+0000","last_peered":"2024-05-27T00:39:28.833971+0000","last_clean":"2024-05-27T00:39:28.833971+0000","last_became_active":"2024-05-27T00:39:25.626973+0000","last_became_peered":"2024-05-27T00:39:25.626973+0000","last_unstale":"2024-05-27T00:39:28.833971+0000","last_undegraded":"2024-05-27T00:39:28.833971+0000","last_fullsized":"2024-05-27T00:39:28.833971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833143+0000","last_change":"2024-05-27T00:39:20.577815+0000","last_active":"2024-05-27T00:39:28.833143+0000","last_peered":"2024-05-27T00:39:28.833143+0000","last_clean":"2024-05-27T00:39:28.833143+0000","last_became_active":"2024-05-27T00:39:20.577495+0000","last_became_peered":"2024-05-27T00:39:20.577495+0000","last_unstale":"2024-05-27T00:39:28.833143+0000","last_undegraded":"2024-05-27T00:39:28.833143+0000","last_fullsized":"2024-05-27T00:39:28.833143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:05.446553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610465+0000","last_change":"2024-05-27T00:39:21.576226+0000","last_active":"2024-05-27T00:39:27.610465+0000","last_peered":"2024-05-27T00:39:27.610465+0000","last_clean":"2024-05-27T00:39:27.610465+0000","last_became_active":"2024-05-27T00:39:21.575940+0000","last_became_peered":"2024-05-27T00:39:21.575940+0000","last_unstale":"2024-05-27T00:39:27.610465+0000","last_undegraded":"2024-05-27T00:39:27.610465+0000","last_fullsized":"2024-05-27T00:39:27.610465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833014+0000","last_change":"2024-05-27T00:39:25.623493+0000","last_active":"2024-05-27T00:39:28.833014+0000","last_peered":"2024-05-27T00:39:28.833014+0000","last_clean":"2024-05-27T00:39:28.833014+0000","last_became_active":"2024-05-27T00:39:25.623101+0000","last_became_peered":"2024-05-27T00:39:25.623101+0000","last_unstale":"2024-05-27T00:39:28.833014+0000","last_undegraded":"2024-05-27T00:39:28.833014+0000","last_fullsized":"2024-05-27T00:39:28.833014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:12.831089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834471+0000","last_change":"2024-05-27T00:39:21.594839+0000","last_active":"2024-05-27T00:39:28.834471+0000","last_peered":"2024-05-27T00:39:28.834471+0000","last_clean":"2024-05-27T00:39:28.834471+0000","last_became_active":"2024-05-27T00:39:21.594428+0000","last_became_peered":"2024-05-27T00:39:21.594428+0000","last_unstale":"2024-05-27T00:39:28.834471+0000","last_undegraded":"2024-05-27T00:39:28.834471+0000","last_fullsized":"2024-05-27T00:39:28.834471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833832+0000","last_change":"2024-05-27T00:39:20.579815+0000","last_active":"2024-05-27T00:39:28.833832+0000","last_peered":"2024-05-27T00:39:28.833832+0000","last_clean":"2024-05-27T00:39:28.833832+0000","last_became_active":"2024-05-27T00:39:20.579507+0000","last_became_peered":"2024-05-27T00:39:20.579507+0000","last_unstale":"2024-05-27T00:39:28.833832+0000","last_undegraded":"2024-05-27T00:39:28.833832+0000","last_fullsized":"2024-05-27T00:39:28.833832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:17.368730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833761+0000","last_change":"2024-05-27T00:39:25.625823+0000","last_active":"2024-05-27T00:39:28.833761+0000","last_peered":"2024-05-27T00:39:28.833761+0000","last_clean":"2024-05-27T00:39:28.833761+0000","last_became_active":"2024-05-27T00:39:25.625378+0000","last_became_peered":"2024-05-27T00:39:25.625378+0000","last_unstale":"2024-05-27T00:39:28.833761+0000","last_undegraded":"2024-05-27T00:39:28.833761+0000","last_fullsized":"2024-05-27T00:39:28.833761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609891+0000","last_change":"2024-05-27T00:39:21.591774+0000","last_active":"2024-05-27T00:39:27.609891+0000","last_peered":"2024-05-27T00:39:27.609891+0000","last_clean":"2024-05-27T00:39:27.609891+0000","last_became_active":"2024-05-27T00:39:21.591413+0000","last_became_peered":"2024-05-27T00:39:21.591413+0000","last_unstale":"2024-05-27T00:39:27.609891+0000","last_undegraded":"2024-05-27T00:39:27.609891+0000","last_fullsized":"2024-05-27T00:39:27.609891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604467+0000","last_change":"2024-05-27T00:39:20.562227+0000","last_active":"2024-05-27T00:39:27.604467+0000","last_peered":"2024-05-27T00:39:27.604467+0000","last_clean":"2024-05-27T00:39:27.604467+0000","last_became_active":"2024-05-27T00:39:20.561938+0000","last_became_peered":"2024-05-27T00:39:20.561938+0000","last_unstale":"2024-05-27T00:39:27.604467+0000","last_undegraded":"2024-05-27T00:39:27.604467+0000","last_fullsized":"2024-05-27T00:39:27.604467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834385+0000","last_change":"2024-05-27T00:39:25.628730+0000","last_active":"2024-05-27T00:39:28.834385+0000","last_peered":"2024-05-27T00:39:28.834385+0000","last_clean":"2024-05-27T00:39:28.834385+0000","last_became_active":"2024-05-27T00:39:25.628352+0000","last_became_peered":"2024-05-27T00:39:25.628352+0000","last_unstale":"2024-05-27T00:39:28.834385+0000","last_undegraded":"2024-05-27T00:39:28.834385+0000","last_fullsized":"2024-05-27T00:39:28.834385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609789+0000","last_change":"2024-05-27T00:39:21.585438+0000","last_active":"2024-05-27T00:39:27.609789+0000","last_peered":"2024-05-27T00:39:27.609789+0000","last_clean":"2024-05-27T00:39:27.609789+0000","last_became_active":"2024-05-27T00:39:21.585062+0000","last_became_peered":"2024-05-27T00:39:21.585062+0000","last_unstale":"2024-05-27T00:39:27.609789+0000","last_undegraded":"2024-05-27T00:39:27.609789+0000","last_fullsized":"2024-05-27T00:39:27.609789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604165+0000","last_change":"2024-05-27T00:39:20.570857+0000","last_active":"2024-05-27T00:39:27.604165+0000","last_peered":"2024-05-27T00:39:27.604165+0000","last_clean":"2024-05-27T00:39:27.604165+0000","last_became_active":"2024-05-27T00:39:20.570564+0000","last_became_peered":"2024-05-27T00:39:20.570564+0000","last_unstale":"2024-05-27T00:39:27.604165+0000","last_undegraded":"2024-05-27T00:39:27.604165+0000","last_fullsized":"2024-05-27T00:39:27.604165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605057+0000","last_change":"2024-05-27T00:39:25.963988+0000","last_active":"2024-05-27T00:39:27.605057+0000","last_peered":"2024-05-27T00:39:27.605057+0000","last_clean":"2024-05-27T00:39:27.605057+0000","last_became_active":"2024-05-27T00:39:25.963532+0000","last_became_peered":"2024-05-27T00:39:25.963532+0000","last_unstale":"2024-05-27T00:39:27.605057+0000","last_undegraded":"2024-05-27T00:39:27.605057+0000","last_fullsized":"2024-05-27T00:39:27.605057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833654+0000","last_change":"2024-05-27T00:39:21.590869+0000","last_active":"2024-05-27T00:39:28.833654+0000","last_peered":"2024-05-27T00:39:28.833654+0000","last_clean":"2024-05-27T00:39:28.833654+0000","last_became_active":"2024-05-27T00:39:21.584136+0000","last_became_peered":"2024-05-27T00:39:21.584136+0000","last_unstale":"2024-05-27T00:39:28.833654+0000","last_undegraded":"2024-05-27T00:39:28.833654+0000","last_fullsized":"2024-05-27T00:39:28.833654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:48.264046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832474+0000","last_change":"2024-05-27T00:39:20.579097+0000","last_active":"2024-05-27T00:39:28.832474+0000","last_peered":"2024-05-27T00:39:28.832474+0000","last_clean":"2024-05-27T00:39:28.832474+0000","last_became_active":"2024-05-27T00:39:20.578796+0000","last_became_peered":"2024-05-27T00:39:20.578796+0000","last_unstale":"2024-05-27T00:39:28.832474+0000","last_undegraded":"2024-05-27T00:39:28.832474+0000","last_fullsized":"2024-05-27T00:39:28.832474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:54.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833500+0000","last_change":"2024-05-27T00:39:25.628322+0000","last_active":"2024-05-27T00:39:28.833500+0000","last_peered":"2024-05-27T00:39:28.833500+0000","last_clean":"2024-05-27T00:39:28.833500+0000","last_became_active":"2024-05-27T00:39:25.627878+0000","last_became_peered":"2024-05-27T00:39:25.627878+0000","last_unstale":"2024-05-27T00:39:28.833500+0000","last_undegraded":"2024-05-27T00:39:28.833500+0000","last_fullsized":"2024-05-27T00:39:28.833500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:42.049691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133677+0000","last_change":"2024-05-27T00:39:21.581752+0000","last_active":"2024-05-27T00:39:28.133677+0000","last_peered":"2024-05-27T00:39:28.133677+0000","last_clean":"2024-05-27T00:39:28.133677+0000","last_became_active":"2024-05-27T00:39:21.581488+0000","last_became_peered":"2024-05-27T00:39:21.581488+0000","last_unstale":"2024-05-27T00:39:28.133677+0000","last_undegraded":"2024-05-27T00:39:28.133677+0000","last_fullsized":"2024-05-27T00:39:28.133677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835187+0000","last_change":"2024-05-27T00:39:20.578119+0000","last_active":"2024-05-27T00:39:28.835187+0000","last_peered":"2024-05-27T00:39:28.835187+0000","last_clean":"2024-05-27T00:39:28.835187+0000","last_became_active":"2024-05-27T00:39:20.577830+0000","last_became_peered":"2024-05-27T00:39:20.577830+0000","last_unstale":"2024-05-27T00:39:28.835187+0000","last_undegraded":"2024-05-27T00:39:28.835187+0000","last_fullsized":"2024-05-27T00:39:28.835187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832957+0000","last_change":"2024-05-27T00:39:25.965993+0000","last_active":"2024-05-27T00:39:28.832957+0000","last_peered":"2024-05-27T00:39:28.832957+0000","last_clean":"2024-05-27T00:39:28.832957+0000","last_became_active":"2024-05-27T00:39:25.965384+0000","last_became_peered":"2024-05-27T00:39:25.965384+0000","last_unstale":"2024-05-27T00:39:28.832957+0000","last_undegraded":"2024-05-27T00:39:28.832957+0000","last_fullsized":"2024-05-27T00:39:28.832957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610354+0000","last_change":"2024-05-27T00:39:21.591396+0000","last_active":"2024-05-27T00:39:27.610354+0000","last_peered":"2024-05-27T00:39:27.610354+0000","last_clean":"2024-05-27T00:39:27.610354+0000","last_became_active":"2024-05-27T00:39:21.591116+0000","last_became_peered":"2024-05-27T00:39:21.591116+0000","last_unstale":"2024-05-27T00:39:27.610354+0000","last_undegraded":"2024-05-27T00:39:27.610354+0000","last_fullsized":"2024-05-27T00:39:27.610354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604797+0000","last_change":"2024-05-27T00:39:20.573838+0000","last_active":"2024-05-27T00:39:27.604797+0000","last_peered":"2024-05-27T00:39:27.604797+0000","last_clean":"2024-05-27T00:39:27.604797+0000","last_became_active":"2024-05-27T00:39:20.573515+0000","last_became_peered":"2024-05-27T00:39:20.573515+0000","last_unstale":"2024-05-27T00:39:27.604797+0000","last_undegraded":"2024-05-27T00:39:27.604797+0000","last_fullsized":"2024-05-27T00:39:27.604797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609614+0000","last_change":"2024-05-27T00:39:25.960920+0000","last_active":"2024-05-27T00:39:27.609614+0000","last_peered":"2024-05-27T00:39:27.609614+0000","last_clean":"2024-05-27T00:39:27.609614+0000","last_became_active":"2024-05-27T00:39:25.960487+0000","last_became_peered":"2024-05-27T00:39:25.960487+0000","last_unstale":"2024-05-27T00:39:27.609614+0000","last_undegraded":"2024-05-27T00:39:27.609614+0000","last_fullsized":"2024-05-27T00:39:27.609614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:59.647950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604641+0000","last_change":"2024-05-27T00:39:21.583980+0000","last_active":"2024-05-27T00:39:27.604641+0000","last_peered":"2024-05-27T00:39:27.604641+0000","last_clean":"2024-05-27T00:39:27.604641+0000","last_became_active":"2024-05-27T00:39:21.583700+0000","last_became_peered":"2024-05-27T00:39:21.583700+0000","last_unstale":"2024-05-27T00:39:27.604641+0000","last_undegraded":"2024-05-27T00:39:27.604641+0000","last_fullsized":"2024-05-27T00:39:27.604641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834010+0000","last_change":"2024-05-27T00:39:20.579198+0000","last_active":"2024-05-27T00:39:28.834010+0000","last_peered":"2024-05-27T00:39:28.834010+0000","last_clean":"2024-05-27T00:39:28.834010+0000","last_became_active":"2024-05-27T00:39:20.578902+0000","last_became_peered":"2024-05-27T00:39:20.578902+0000","last_unstale":"2024-05-27T00:39:28.834010+0000","last_undegraded":"2024-05-27T00:39:28.834010+0000","last_fullsized":"2024-05-27T00:39:28.834010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.610865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.962036+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.961152+0000","last_became_peered":"2024-05-27T00:39:25.961152+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610557+0000","last_change":"2024-05-27T00:39:21.591906+0000","last_active":"2024-05-27T00:39:27.610557+0000","last_peered":"2024-05-27T00:39:27.610557+0000","last_clean":"2024-05-27T00:39:27.610557+0000","last_became_active":"2024-05-27T00:39:21.591611+0000","last_became_peered":"2024-05-27T00:39:21.591611+0000","last_unstale":"2024-05-27T00:39:27.610557+0000","last_undegraded":"2024-05-27T00:39:27.610557+0000","last_fullsized":"2024-05-27T00:39:27.610557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604735+0000","last_change":"2024-05-27T00:39:20.569834+0000","last_active":"2024-05-27T00:39:27.604735+0000","last_peered":"2024-05-27T00:39:27.604735+0000","last_clean":"2024-05-27T00:39:27.604735+0000","last_became_active":"2024-05-27T00:39:20.569540+0000","last_became_peered":"2024-05-27T00:39:20.569540+0000","last_unstale":"2024-05-27T00:39:27.604735+0000","last_undegraded":"2024-05-27T00:39:27.604735+0000","last_fullsized":"2024-05-27T00:39:27.604735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604262+0000","last_change":"2024-05-27T00:39:25.627533+0000","last_active":"2024-05-27T00:39:27.604262+0000","last_peered":"2024-05-27T00:39:27.604262+0000","last_clean":"2024-05-27T00:39:27.604262+0000","last_became_active":"2024-05-27T00:39:25.625872+0000","last_became_peered":"2024-05-27T00:39:25.625872+0000","last_unstale":"2024-05-27T00:39:27.604262+0000","last_undegraded":"2024-05-27T00:39:27.604262+0000","last_fullsized":"2024-05-27T00:39:27.604262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:17.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834370+0000","last_change":"2024-05-27T00:39:21.598984+0000","last_active":"2024-05-27T00:39:28.834370+0000","last_peered":"2024-05-27T00:39:28.834370+0000","last_clean":"2024-05-27T00:39:28.834370+0000","last_became_active":"2024-05-27T00:39:21.594222+0000","last_became_peered":"2024-05-27T00:39:21.594222+0000","last_unstale":"2024-05-27T00:39:28.834370+0000","last_undegraded":"2024-05-27T00:39:28.834370+0000","last_fullsized":"2024-05-27T00:39:28.834370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610187+0000","last_change":"2024-05-27T00:39:20.565350+0000","last_active":"2024-05-27T00:39:27.610187+0000","last_peered":"2024-05-27T00:39:27.610187+0000","last_clean":"2024-05-27T00:39:27.610187+0000","last_became_active":"2024-05-27T00:39:20.562112+0000","last_became_peered":"2024-05-27T00:39:20.562112+0000","last_unstale":"2024-05-27T00:39:27.610187+0000","last_undegraded":"2024-05-27T00:39:27.610187+0000","last_fullsized":"2024-05-27T00:39:27.610187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604681+0000","last_change":"2024-05-27T00:39:25.624845+0000","last_active":"2024-05-27T00:39:27.604681+0000","last_peered":"2024-05-27T00:39:27.604681+0000","last_clean":"2024-05-27T00:39:27.604681+0000","last_became_active":"2024-05-27T00:39:25.624336+0000","last_became_peered":"2024-05-27T00:39:25.624336+0000","last_unstale":"2024-05-27T00:39:27.604681+0000","last_undegraded":"2024-05-27T00:39:27.604681+0000","last_fullsized":"2024-05-27T00:39:27.604681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:18:01.988259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604350+0000","last_change":"2024-05-27T00:39:21.586786+0000","last_active":"2024-05-27T00:39:27.604350+0000","last_peered":"2024-05-27T00:39:27.604350+0000","last_clean":"2024-05-27T00:39:27.604350+0000","last_became_active":"2024-05-27T00:39:21.586521+0000","last_became_peered":"2024-05-27T00:39:21.586521+0000","last_unstale":"2024-05-27T00:39:27.604350+0000","last_undegraded":"2024-05-27T00:39:27.604350+0000","last_fullsized":"2024-05-27T00:39:27.604350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573104+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.572665+0000","last_became_peered":"2024-05-27T00:39:20.572665+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.605183+0000","last_change":"2024-05-27T00:39:25.625105+0000","last_active":"2024-05-27T00:39:27.605183+0000","last_peered":"2024-05-27T00:39:27.605183+0000","last_clean":"2024-05-27T00:39:27.605183+0000","last_became_active":"2024-05-27T00:39:25.624400+0000","last_became_peered":"2024-05-27T00:39:25.624400+0000","last_unstale":"2024-05-27T00:39:27.605183+0000","last_undegraded":"2024-05-27T00:39:27.605183+0000","last_fullsized":"2024-05-27T00:39:27.605183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133189+0000","last_change":"2024-05-27T00:39:21.578555+0000","last_active":"2024-05-27T00:39:28.133189+0000","last_peered":"2024-05-27T00:39:28.133189+0000","last_clean":"2024-05-27T00:39:28.133189+0000","last_became_active":"2024-05-27T00:39:21.578291+0000","last_became_peered":"2024-05-27T00:39:21.578291+0000","last_unstale":"2024-05-27T00:39:28.133189+0000","last_undegraded":"2024-05-27T00:39:28.133189+0000","last_fullsized":"2024-05-27T00:39:28.133189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:00.616291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832328+0000","last_change":"2024-05-27T00:39:20.578837+0000","last_active":"2024-05-27T00:39:28.832328+0000","last_peered":"2024-05-27T00:39:28.832328+0000","last_clean":"2024-05-27T00:39:28.832328+0000","last_became_active":"2024-05-27T00:39:20.578414+0000","last_became_peered":"2024-05-27T00:39:20.578414+0000","last_unstale":"2024-05-27T00:39:28.832328+0000","last_undegraded":"2024-05-27T00:39:28.832328+0000","last_fullsized":"2024-05-27T00:39:28.832328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:32.759982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133680+0000","last_change":"2024-05-27T00:39:25.962038+0000","last_active":"2024-05-27T00:39:28.133680+0000","last_peered":"2024-05-27T00:39:28.133680+0000","last_clean":"2024-05-27T00:39:28.133680+0000","last_became_active":"2024-05-27T00:39:25.961489+0000","last_became_peered":"2024-05-27T00:39:25.961489+0000","last_unstale":"2024-05-27T00:39:28.133680+0000","last_undegraded":"2024-05-27T00:39:28.133680+0000","last_fullsized":"2024-05-27T00:39:28.133680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:36.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609779+0000","last_change":"2024-05-27T00:39:21.593420+0000","last_active":"2024-05-27T00:39:27.609779+0000","last_peered":"2024-05-27T00:39:27.609779+0000","last_clean":"2024-05-27T00:39:27.609779+0000","last_became_active":"2024-05-27T00:39:21.593165+0000","last_became_peered":"2024-05-27T00:39:21.593165+0000","last_unstale":"2024-05-27T00:39:27.609779+0000","last_undegraded":"2024-05-27T00:39:27.609779+0000","last_fullsized":"2024-05-27T00:39:27.609779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610340+0000","last_change":"2024-05-27T00:39:20.573661+0000","last_active":"2024-05-27T00:39:27.610340+0000","last_peered":"2024-05-27T00:39:27.610340+0000","last_clean":"2024-05-27T00:39:27.610340+0000","last_became_active":"2024-05-27T00:39:20.572675+0000","last_became_peered":"2024-05-27T00:39:20.572675+0000","last_unstale":"2024-05-27T00:39:27.610340+0000","last_undegraded":"2024-05-27T00:39:27.610340+0000","last_fullsized":"2024-05-27T00:39:27.610340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832965+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:39:28.832965+0000","last_peered":"2024-05-27T00:39:28.832965+0000","last_clean":"2024-05-27T00:39:28.832965+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:39:28.832965+0000","last_undegraded":"2024-05-27T00:39:28.832965+0000","last_fullsized":"2024-05-27T00:39:28.832965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834052+0000","last_change":"2024-05-27T00:39:25.964404+0000","last_active":"2024-05-27T00:39:28.834052+0000","last_peered":"2024-05-27T00:39:28.834052+0000","last_clean":"2024-05-27T00:39:28.834052+0000","last_became_active":"2024-05-27T00:39:25.963955+0000","last_became_peered":"2024-05-27T00:39:25.963955+0000","last_unstale":"2024-05-27T00:39:28.834052+0000","last_undegraded":"2024-05-27T00:39:28.834052+0000","last_fullsized":"2024-05-27T00:39:28.834052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:48.698768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834325+0000","last_change":"2024-05-27T00:39:21.585811+0000","last_active":"2024-05-27T00:39:28.834325+0000","last_peered":"2024-05-27T00:39:28.834325+0000","last_clean":"2024-05-27T00:39:28.834325+0000","last_became_active":"2024-05-27T00:39:21.585466+0000","last_became_peered":"2024-05-27T00:39:21.585466+0000","last_unstale":"2024-05-27T00:39:28.834325+0000","last_undegraded":"2024-05-27T00:39:28.834325+0000","last_fullsized":"2024-05-27T00:39:28.834325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.582020+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.581718+0000","last_became_peered":"2024-05-27T00:39:20.581718+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834411+0000","last_change":"2024-05-27T00:39:25.961052+0000","last_active":"2024-05-27T00:39:28.834411+0000","last_peered":"2024-05-27T00:39:28.834411+0000","last_clean":"2024-05-27T00:39:28.834411+0000","last_became_active":"2024-05-27T00:39:25.960632+0000","last_became_peered":"2024-05-27T00:39:25.960632+0000","last_unstale":"2024-05-27T00:39:28.834411+0000","last_undegraded":"2024-05-27T00:39:28.834411+0000","last_fullsized":"2024-05-27T00:39:28.834411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610612+0000","last_change":"2024-05-27T00:39:21.587149+0000","last_active":"2024-05-27T00:39:27.610612+0000","last_peered":"2024-05-27T00:39:27.610612+0000","last_clean":"2024-05-27T00:39:27.610612+0000","last_became_active":"2024-05-27T00:39:21.585877+0000","last_became_peered":"2024-05-27T00:39:21.585877+0000","last_unstale":"2024-05-27T00:39:27.610612+0000","last_undegraded":"2024-05-27T00:39:27.610612+0000","last_fullsized":"2024-05-27T00:39:27.610612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604682+0000","last_change":"2024-05-27T00:39:20.575827+0000","last_active":"2024-05-27T00:39:27.604682+0000","last_peered":"2024-05-27T00:39:27.604682+0000","last_clean":"2024-05-27T00:39:27.604682+0000","last_became_active":"2024-05-27T00:39:20.575555+0000","last_became_peered":"2024-05-27T00:39:20.575555+0000","last_unstale":"2024-05-27T00:39:27.604682+0000","last_undegraded":"2024-05-27T00:39:27.604682+0000","last_fullsized":"2024-05-27T00:39:27.604682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133662+0000","last_change":"2024-05-27T00:39:25.626022+0000","last_active":"2024-05-27T00:39:28.133662+0000","last_peered":"2024-05-27T00:39:28.133662+0000","last_clean":"2024-05-27T00:39:28.133662+0000","last_became_active":"2024-05-27T00:39:25.625643+0000","last_became_peered":"2024-05-27T00:39:25.625643+0000","last_unstale":"2024-05-27T00:39:28.133662+0000","last_undegraded":"2024-05-27T00:39:28.133662+0000","last_fullsized":"2024-05-27T00:39:28.133662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833753+0000","last_change":"2024-05-27T00:39:21.599339+0000","last_active":"2024-05-27T00:39:28.833753+0000","last_peered":"2024-05-27T00:39:28.833753+0000","last_clean":"2024-05-27T00:39:28.833753+0000","last_became_active":"2024-05-27T00:39:21.598994+0000","last_became_peered":"2024-05-27T00:39:21.598994+0000","last_unstale":"2024-05-27T00:39:28.833753+0000","last_undegraded":"2024-05-27T00:39:28.833753+0000","last_fullsized":"2024-05-27T00:39:28.833753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:04:13.702562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831770+0000","last_change":"2024-05-27T00:39:20.572771+0000","last_active":"2024-05-27T00:39:28.831770+0000","last_peered":"2024-05-27T00:39:28.831770+0000","last_clean":"2024-05-27T00:39:28.831770+0000","last_became_active":"2024-05-27T00:39:20.572486+0000","last_became_peered":"2024-05-27T00:39:20.572486+0000","last_unstale":"2024-05-27T00:39:28.831770+0000","last_undegraded":"2024-05-27T00:39:28.831770+0000","last_fullsized":"2024-05-27T00:39:28.831770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609790+0000","last_change":"2024-05-27T00:39:25.962926+0000","last_active":"2024-05-27T00:39:27.609790+0000","last_peered":"2024-05-27T00:39:27.609790+0000","last_clean":"2024-05-27T00:39:27.609790+0000","last_became_active":"2024-05-27T00:39:25.962471+0000","last_became_peered":"2024-05-27T00:39:25.962471+0000","last_unstale":"2024-05-27T00:39:27.609790+0000","last_undegraded":"2024-05-27T00:39:27.609790+0000","last_fullsized":"2024-05-27T00:39:27.609790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833146+0000","last_change":"2024-05-27T00:39:21.596571+0000","last_active":"2024-05-27T00:39:28.833146+0000","last_peered":"2024-05-27T00:39:28.833146+0000","last_clean":"2024-05-27T00:39:28.833146+0000","last_became_active":"2024-05-27T00:39:21.596278+0000","last_became_peered":"2024-05-27T00:39:21.596278+0000","last_unstale":"2024-05-27T00:39:28.833146+0000","last_undegraded":"2024-05-27T00:39:28.833146+0000","last_fullsized":"2024-05-27T00:39:28.833146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:42.881822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832599+0000","last_change":"2024-05-27T00:39:20.575461+0000","last_active":"2024-05-27T00:39:28.832599+0000","last_peered":"2024-05-27T00:39:28.832599+0000","last_clean":"2024-05-27T00:39:28.832599+0000","last_became_active":"2024-05-27T00:39:20.575168+0000","last_became_peered":"2024-05-27T00:39:20.575168+0000","last_unstale":"2024-05-27T00:39:28.832599+0000","last_undegraded":"2024-05-27T00:39:28.832599+0000","last_fullsized":"2024-05-27T00:39:28.832599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:04:51.312690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832581+0000","last_change":"2024-05-27T00:39:25.627556+0000","last_active":"2024-05-27T00:39:28.832581+0000","last_peered":"2024-05-27T00:39:28.832581+0000","last_clean":"2024-05-27T00:39:28.832581+0000","last_became_active":"2024-05-27T00:39:25.627169+0000","last_became_peered":"2024-05-27T00:39:25.627169+0000","last_unstale":"2024-05-27T00:39:28.832581+0000","last_undegraded":"2024-05-27T00:39:28.832581+0000","last_fullsized":"2024-05-27T00:39:28.832581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:44.835484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832699+0000","last_change":"2024-05-27T00:39:21.596905+0000","last_active":"2024-05-27T00:39:28.832699+0000","last_peered":"2024-05-27T00:39:28.832699+0000","last_clean":"2024-05-27T00:39:28.832699+0000","last_became_active":"2024-05-27T00:39:21.596586+0000","last_became_peered":"2024-05-27T00:39:21.596586+0000","last_unstale":"2024-05-27T00:39:28.832699+0000","last_undegraded":"2024-05-27T00:39:28.832699+0000","last_fullsized":"2024-05-27T00:39:28.832699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831766+0000","last_change":"2024-05-27T00:39:20.573588+0000","last_active":"2024-05-27T00:39:28.831766+0000","last_peered":"2024-05-27T00:39:28.831766+0000","last_clean":"2024-05-27T00:39:28.831766+0000","last_became_active":"2024-05-27T00:39:20.573302+0000","last_became_peered":"2024-05-27T00:39:20.573302+0000","last_unstale":"2024-05-27T00:39:28.831766+0000","last_undegraded":"2024-05-27T00:39:28.831766+0000","last_fullsized":"2024-05-27T00:39:28.831766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133542+0000","last_change":"2024-05-27T00:39:25.620609+0000","last_active":"2024-05-27T00:39:28.133542+0000","last_peered":"2024-05-27T00:39:28.133542+0000","last_clean":"2024-05-27T00:39:28.133542+0000","last_became_active":"2024-05-27T00:39:25.620185+0000","last_became_peered":"2024-05-27T00:39:25.620185+0000","last_unstale":"2024-05-27T00:39:28.133542+0000","last_undegraded":"2024-05-27T00:39:28.133542+0000","last_fullsized":"2024-05-27T00:39:28.133542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835033+0000","last_change":"2024-05-27T00:39:21.593089+0000","last_active":"2024-05-27T00:39:28.835033+0000","last_peered":"2024-05-27T00:39:28.835033+0000","last_clean":"2024-05-27T00:39:28.835033+0000","last_became_active":"2024-05-27T00:39:21.592034+0000","last_became_peered":"2024-05-27T00:39:21.592034+0000","last_unstale":"2024-05-27T00:39:28.835033+0000","last_undegraded":"2024-05-27T00:39:28.835033+0000","last_fullsized":"2024-05-27T00:39:28.835033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834807+0000","last_change":"2024-05-27T00:39:20.576289+0000","last_active":"2024-05-27T00:39:28.834807+0000","last_peered":"2024-05-27T00:39:28.834807+0000","last_clean":"2024-05-27T00:39:28.834807+0000","last_became_active":"2024-05-27T00:39:20.576008+0000","last_became_peered":"2024-05-27T00:39:20.576008+0000","last_unstale":"2024-05-27T00:39:28.834807+0000","last_undegraded":"2024-05-27T00:39:28.834807+0000","last_fullsized":"2024-05-27T00:39:28.834807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610207+0000","last_change":"2024-05-27T00:39:25.964215+0000","last_active":"2024-05-27T00:39:27.610207+0000","last_peered":"2024-05-27T00:39:27.610207+0000","last_clean":"2024-05-27T00:39:27.610207+0000","last_became_active":"2024-05-27T00:39:25.963695+0000","last_became_peered":"2024-05-27T00:39:25.963695+0000","last_unstale":"2024-05-27T00:39:27.610207+0000","last_undegraded":"2024-05-27T00:39:27.610207+0000","last_fullsized":"2024-05-27T00:39:27.610207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:00:27.182513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833721+0000","last_change":"2024-05-27T00:39:21.599920+0000","last_active":"2024-05-27T00:39:28.833721+0000","last_peered":"2024-05-27T00:39:28.833721+0000","last_clean":"2024-05-27T00:39:28.833721+0000","last_became_active":"2024-05-27T00:39:21.597737+0000","last_became_peered":"2024-05-27T00:39:21.597737+0000","last_unstale":"2024-05-27T00:39:28.833721+0000","last_undegraded":"2024-05-27T00:39:28.833721+0000","last_fullsized":"2024-05-27T00:39:28.833721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:34:38.014686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610101+0000","last_change":"2024-05-27T00:39:20.571178+0000","last_active":"2024-05-27T00:39:27.610101+0000","last_peered":"2024-05-27T00:39:27.610101+0000","last_clean":"2024-05-27T00:39:27.610101+0000","last_became_active":"2024-05-27T00:39:20.570884+0000","last_became_peered":"2024-05-27T00:39:20.570884+0000","last_unstale":"2024-05-27T00:39:27.610101+0000","last_undegraded":"2024-05-27T00:39:27.610101+0000","last_fullsized":"2024-05-27T00:39:27.610101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610204+0000","last_change":"2024-05-27T00:39:25.961014+0000","last_active":"2024-05-27T00:39:27.610204+0000","last_peered":"2024-05-27T00:39:27.610204+0000","last_clean":"2024-05-27T00:39:27.610204+0000","last_became_active":"2024-05-27T00:39:25.960630+0000","last_became_peered":"2024-05-27T00:39:25.960630+0000","last_unstale":"2024-05-27T00:39:27.610204+0000","last_undegraded":"2024-05-27T00:39:27.610204+0000","last_fullsized":"2024-05-27T00:39:27.610204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:53:07.966829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133320+0000","last_change":"2024-05-27T00:39:21.583015+0000","last_active":"2024-05-27T00:39:28.133320+0000","last_peered":"2024-05-27T00:39:28.133320+0000","last_clean":"2024-05-27T00:39:28.133320+0000","last_became_active":"2024-05-27T00:39:21.582722+0000","last_became_peered":"2024-05-27T00:39:21.582722+0000","last_unstale":"2024-05-27T00:39:28.133320+0000","last_undegraded":"2024-05-27T00:39:28.133320+0000","last_fullsized":"2024-05-27T00:39:28.133320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:04.988100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133718+0000","last_change":"2024-05-27T00:39:20.575435+0000","last_active":"2024-05-27T00:39:28.133718+0000","last_peered":"2024-05-27T00:39:28.133718+0000","last_clean":"2024-05-27T00:39:28.133718+0000","last_became_active":"2024-05-27T00:39:20.575144+0000","last_became_peered":"2024-05-27T00:39:20.575144+0000","last_unstale":"2024-05-27T00:39:28.133718+0000","last_undegraded":"2024-05-27T00:39:28.133718+0000","last_fullsized":"2024-05-27T00:39:28.133718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:25:38.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610849+0000","last_change":"2024-05-27T00:39:25.961559+0000","last_active":"2024-05-27T00:39:27.610849+0000","last_peered":"2024-05-27T00:39:27.610849+0000","last_clean":"2024-05-27T00:39:27.610849+0000","last_became_active":"2024-05-27T00:39:25.961155+0000","last_became_peered":"2024-05-27T00:39:25.961155+0000","last_unstale":"2024-05-27T00:39:27.610849+0000","last_undegraded":"2024-05-27T00:39:27.610849+0000","last_fullsized":"2024-05-27T00:39:27.610849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:52:13.747875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834212+0000","last_change":"2024-05-27T00:39:21.593738+0000","last_active":"2024-05-27T00:39:28.834212+0000","last_peered":"2024-05-27T00:39:28.834212+0000","last_clean":"2024-05-27T00:39:28.834212+0000","last_became_active":"2024-05-27T00:39:21.591298+0000","last_became_peered":"2024-05-27T00:39:21.591298+0000","last_unstale":"2024-05-27T00:39:28.834212+0000","last_undegraded":"2024-05-27T00:39:28.834212+0000","last_fullsized":"2024-05-27T00:39:28.834212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:28.484267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:32.251423+0000","last_change":"2024-05-27T00:39:20.577937+0000","last_active":"2024-05-27T00:39:32.251423+0000","last_peered":"2024-05-27T00:39:32.251423+0000","last_clean":"2024-05-27T00:39:32.251423+0000","last_became_active":"2024-05-27T00:39:20.577630+0000","last_became_peered":"2024-05-27T00:39:20.577630+0000","last_unstale":"2024-05-27T00:39:32.251423+0000","last_undegraded":"2024-05-27T00:39:32.251423+0000","last_fullsized":"2024-05-27T00:39:32.251423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604388+0000","last_change":"2024-05-27T00:39:25.620344+0000","last_active":"2024-05-27T00:39:27.604388+0000","last_peered":"2024-05-27T00:39:27.604388+0000","last_clean":"2024-05-27T00:39:27.604388+0000","last_became_active":"2024-05-27T00:39:25.618090+0000","last_became_peered":"2024-05-27T00:39:25.618090+0000","last_unstale":"2024-05-27T00:39:27.604388+0000","last_undegraded":"2024-05-27T00:39:27.604388+0000","last_fullsized":"2024-05-27T00:39:27.604388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:46.916828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834698+0000","last_change":"2024-05-27T00:39:21.593265+0000","last_active":"2024-05-27T00:39:28.834698+0000","last_peered":"2024-05-27T00:39:28.834698+0000","last_clean":"2024-05-27T00:39:28.834698+0000","last_became_active":"2024-05-27T00:39:21.592984+0000","last_became_peered":"2024-05-27T00:39:21.592984+0000","last_unstale":"2024-05-27T00:39:28.834698+0000","last_undegraded":"2024-05-27T00:39:28.834698+0000","last_fullsized":"2024-05-27T00:39:28.834698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:06.451979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604736+0000","last_change":"2024-05-27T00:39:20.570519+0000","last_active":"2024-05-27T00:39:27.604736+0000","last_peered":"2024-05-27T00:39:27.604736+0000","last_clean":"2024-05-27T00:39:27.604736+0000","last_became_active":"2024-05-27T00:39:20.570230+0000","last_became_peered":"2024-05-27T00:39:20.570230+0000","last_unstale":"2024-05-27T00:39:27.604736+0000","last_undegraded":"2024-05-27T00:39:27.604736+0000","last_fullsized":"2024-05-27T00:39:27.604736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:47.258620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834196+0000","last_change":"2024-05-27T00:39:25.628554+0000","last_active":"2024-05-27T00:39:28.834196+0000","last_peered":"2024-05-27T00:39:28.834196+0000","last_clean":"2024-05-27T00:39:28.834196+0000","last_became_active":"2024-05-27T00:39:25.627971+0000","last_became_peered":"2024-05-27T00:39:25.627971+0000","last_unstale":"2024-05-27T00:39:28.834196+0000","last_undegraded":"2024-05-27T00:39:28.834196+0000","last_fullsized":"2024-05-27T00:39:28.834196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133800+0000","last_change":"2024-05-27T00:39:21.576653+0000","last_active":"2024-05-27T00:39:28.133800+0000","last_peered":"2024-05-27T00:39:28.133800+0000","last_clean":"2024-05-27T00:39:28.133800+0000","last_became_active":"2024-05-27T00:39:21.576383+0000","last_became_peered":"2024-05-27T00:39:21.576383+0000","last_unstale":"2024-05-27T00:39:28.133800+0000","last_undegraded":"2024-05-27T00:39:28.133800+0000","last_fullsized":"2024-05-27T00:39:28.133800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610702+0000","last_change":"2024-05-27T00:39:20.575867+0000","last_active":"2024-05-27T00:39:27.610702+0000","last_peered":"2024-05-27T00:39:27.610702+0000","last_clean":"2024-05-27T00:39:27.610702+0000","last_became_active":"2024-05-27T00:39:20.575557+0000","last_became_peered":"2024-05-27T00:39:20.575557+0000","last_unstale":"2024-05-27T00:39:27.610702+0000","last_undegraded":"2024-05-27T00:39:27.610702+0000","last_fullsized":"2024-05-27T00:39:27.610702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609795+0000","last_change":"2024-05-27T00:39:25.627020+0000","last_active":"2024-05-27T00:39:27.609795+0000","last_peered":"2024-05-27T00:39:27.609795+0000","last_clean":"2024-05-27T00:39:27.609795+0000","last_became_active":"2024-05-27T00:39:25.626629+0000","last_became_peered":"2024-05-27T00:39:25.626629+0000","last_unstale":"2024-05-27T00:39:27.609795+0000","last_undegraded":"2024-05-27T00:39:27.609795+0000","last_fullsized":"2024-05-27T00:39:27.609795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:51:57.470204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610617+0000","last_change":"2024-05-27T00:39:21.584614+0000","last_active":"2024-05-27T00:39:27.610617+0000","last_peered":"2024-05-27T00:39:27.610617+0000","last_clean":"2024-05-27T00:39:27.610617+0000","last_became_active":"2024-05-27T00:39:21.584349+0000","last_became_peered":"2024-05-27T00:39:21.584349+0000","last_unstale":"2024-05-27T00:39:27.610617+0000","last_undegraded":"2024-05-27T00:39:27.610617+0000","last_fullsized":"2024-05-27T00:39:27.610617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834142+0000","last_change":"2024-05-27T00:39:20.575901+0000","last_active":"2024-05-27T00:39:28.834142+0000","last_peered":"2024-05-27T00:39:28.834142+0000","last_clean":"2024-05-27T00:39:28.834142+0000","last_became_active":"2024-05-27T00:39:20.575521+0000","last_became_peered":"2024-05-27T00:39:20.575521+0000","last_unstale":"2024-05-27T00:39:28.834142+0000","last_undegraded":"2024-05-27T00:39:28.834142+0000","last_fullsized":"2024-05-27T00:39:28.834142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:53.632216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609706+0000","last_change":"2024-05-27T00:39:25.961740+0000","last_active":"2024-05-27T00:39:27.609706+0000","last_peered":"2024-05-27T00:39:27.609706+0000","last_clean":"2024-05-27T00:39:27.609706+0000","last_became_active":"2024-05-27T00:39:25.960524+0000","last_became_peered":"2024-05-27T00:39:25.960524+0000","last_unstale":"2024-05-27T00:39:27.609706+0000","last_undegraded":"2024-05-27T00:39:27.609706+0000","last_fullsized":"2024-05-27T00:39:27.609706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:38:40.065860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133581+0000","last_change":"2024-05-27T00:39:25.961849+0000","last_active":"2024-05-27T00:39:28.133581+0000","last_peered":"2024-05-27T00:39:28.133581+0000","last_clean":"2024-05-27T00:39:28.133581+0000","last_became_active":"2024-05-27T00:39:25.960998+0000","last_became_peered":"2024-05-27T00:39:25.960998+0000","last_unstale":"2024-05-27T00:39:28.133581+0000","last_undegraded":"2024-05-27T00:39:28.133581+0000","last_fullsized":"2024-05-27T00:39:28.133581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:45.449710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604056+0000","last_change":"2024-05-27T00:39:21.590501+0000","last_active":"2024-05-27T00:39:27.604056+0000","last_peered":"2024-05-27T00:39:27.604056+0000","last_clean":"2024-05-27T00:39:27.604056+0000","last_became_active":"2024-05-27T00:39:21.590246+0000","last_became_peered":"2024-05-27T00:39:21.590246+0000","last_unstale":"2024-05-27T00:39:27.604056+0000","last_undegraded":"2024-05-27T00:39:27.604056+0000","last_fullsized":"2024-05-27T00:39:27.604056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609915+0000","last_change":"2024-05-27T00:39:20.574994+0000","last_active":"2024-05-27T00:39:27.609915+0000","last_peered":"2024-05-27T00:39:27.609915+0000","last_clean":"2024-05-27T00:39:27.609915+0000","last_became_active":"2024-05-27T00:39:20.574713+0000","last_became_peered":"2024-05-27T00:39:20.574713+0000","last_unstale":"2024-05-27T00:39:27.609915+0000","last_undegraded":"2024-05-27T00:39:27.609915+0000","last_fullsized":"2024-05-27T00:39:27.609915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:22:39.485065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835197+0000","last_change":"2024-05-27T00:39:25.629723+0000","last_active":"2024-05-27T00:39:28.835197+0000","last_peered":"2024-05-27T00:39:28.835197+0000","last_clean":"2024-05-27T00:39:28.835197+0000","last_became_active":"2024-05-27T00:39:25.628168+0000","last_became_peered":"2024-05-27T00:39:25.628168+0000","last_unstale":"2024-05-27T00:39:28.835197+0000","last_undegraded":"2024-05-27T00:39:28.835197+0000","last_fullsized":"2024-05-27T00:39:28.835197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:47.576849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609695+0000","last_change":"2024-05-27T00:39:21.591614+0000","last_active":"2024-05-27T00:39:27.609695+0000","last_peered":"2024-05-27T00:39:27.609695+0000","last_clean":"2024-05-27T00:39:27.609695+0000","last_became_active":"2024-05-27T00:39:21.591359+0000","last_became_peered":"2024-05-27T00:39:21.591359+0000","last_unstale":"2024-05-27T00:39:27.609695+0000","last_undegraded":"2024-05-27T00:39:27.609695+0000","last_fullsized":"2024-05-27T00:39:27.609695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610097+0000","last_change":"2024-05-27T00:39:20.575532+0000","last_active":"2024-05-27T00:39:27.610097+0000","last_peered":"2024-05-27T00:39:27.610097+0000","last_clean":"2024-05-27T00:39:27.610097+0000","last_became_active":"2024-05-27T00:39:20.575201+0000","last_became_peered":"2024-05-27T00:39:20.575201+0000","last_unstale":"2024-05-27T00:39:27.610097+0000","last_undegraded":"2024-05-27T00:39:27.610097+0000","last_fullsized":"2024-05-27T00:39:27.610097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:49.707196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834640+0000","last_change":"2024-05-27T00:39:20.577031+0000","last_active":"2024-05-27T00:39:28.834640+0000","last_peered":"2024-05-27T00:39:28.834640+0000","last_clean":"2024-05-27T00:39:28.834640+0000","last_became_active":"2024-05-27T00:39:20.576619+0000","last_became_peered":"2024-05-27T00:39:20.576619+0000","last_unstale":"2024-05-27T00:39:28.834640+0000","last_undegraded":"2024-05-27T00:39:28.834640+0000","last_fullsized":"2024-05-27T00:39:28.834640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:46.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133684+0000","last_change":"2024-05-27T00:39:21.580671+0000","last_active":"2024-05-27T00:39:28.133684+0000","last_peered":"2024-05-27T00:39:28.133684+0000","last_clean":"2024-05-27T00:39:28.133684+0000","last_became_active":"2024-05-27T00:39:21.580434+0000","last_became_peered":"2024-05-27T00:39:21.580434+0000","last_unstale":"2024-05-27T00:39:28.133684+0000","last_undegraded":"2024-05-27T00:39:28.133684+0000","last_fullsized":"2024-05-27T00:39:28.133684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832153+0000","last_change":"2024-05-27T00:39:25.624658+0000","last_active":"2024-05-27T00:39:28.832153+0000","last_peered":"2024-05-27T00:39:28.832153+0000","last_clean":"2024-05-27T00:39:28.832153+0000","last_became_active":"2024-05-27T00:39:25.624289+0000","last_became_peered":"2024-05-27T00:39:25.624289+0000","last_unstale":"2024-05-27T00:39:28.832153+0000","last_undegraded":"2024-05-27T00:39:28.832153+0000","last_fullsized":"2024-05-27T00:39:28.832153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:22.409161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609979+0000","last_change":"2024-05-27T00:39:20.572852+0000","last_active":"2024-05-27T00:39:27.609979+0000","last_peered":"2024-05-27T00:39:27.609979+0000","last_clean":"2024-05-27T00:39:27.609979+0000","last_became_active":"2024-05-27T00:39:20.572562+0000","last_became_peered":"2024-05-27T00:39:20.572562+0000","last_unstale":"2024-05-27T00:39:27.609979+0000","last_undegraded":"2024-05-27T00:39:27.609979+0000","last_fullsized":"2024-05-27T00:39:27.609979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:19.546477+0000","last_clean_scrub_stamp":"2024-05-27T00:39:19.546477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:16.760704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833202+0000","last_change":"2024-05-27T00:39:21.587846+0000","last_active":"2024-05-27T00:39:28.833202+0000","last_peered":"2024-05-27T00:39:28.833202+0000","last_clean":"2024-05-27T00:39:28.833202+0000","last_became_active":"2024-05-27T00:39:21.587579+0000","last_became_peered":"2024-05-27T00:39:21.587579+0000","last_unstale":"2024-05-27T00:39:28.833202+0000","last_undegraded":"2024-05-27T00:39:28.833202+0000","last_fullsized":"2024-05-27T00:39:28.833202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:47.382970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610160+0000","last_change":"2024-05-27T00:39:25.628905+0000","last_active":"2024-05-27T00:39:27.610160+0000","last_peered":"2024-05-27T00:39:27.610160+0000","last_clean":"2024-05-27T00:39:27.610160+0000","last_became_active":"2024-05-27T00:39:25.628229+0000","last_became_peered":"2024-05-27T00:39:25.628229+0000","last_unstale":"2024-05-27T00:39:27.610160+0000","last_undegraded":"2024-05-27T00:39:27.610160+0000","last_fullsized":"2024-05-27T00:39:27.610160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:24.586908+0000","last_clean_scrub_stamp":"2024-05-27T00:39:24.586908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:40:24.502057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834905+0000","last_change":"2024-05-27T00:39:21.593542+0000","last_active":"2024-05-27T00:39:28.834905+0000","last_peered":"2024-05-27T00:39:28.834905+0000","last_clean":"2024-05-27T00:39:28.834905+0000","last_became_active":"2024-05-27T00:39:21.592466+0000","last_became_peered":"2024-05-27T00:39:21.592466+0000","last_unstale":"2024-05-27T00:39:28.834905+0000","last_undegraded":"2024-05-27T00:39:28.834905+0000","last_fullsized":"2024-05-27T00:39:28.834905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.831986+0000","last_change":"2024-05-27T00:39:21.586207+0000","last_active":"2024-05-27T00:39:28.831986+0000","last_peered":"2024-05-27T00:39:28.831986+0000","last_clean":"2024-05-27T00:39:28.831986+0000","last_became_active":"2024-05-27T00:39:21.585899+0000","last_became_peered":"2024-05-27T00:39:21.585899+0000","last_unstale":"2024-05-27T00:39:28.831986+0000","last_undegraded":"2024-05-27T00:39:28.831986+0000","last_fullsized":"2024-05-27T00:39:28.831986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:26:55.339912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610077+0000","last_change":"2024-05-27T00:39:21.589304+0000","last_active":"2024-05-27T00:39:27.610077+0000","last_peered":"2024-05-27T00:39:27.610077+0000","last_clean":"2024-05-27T00:39:27.610077+0000","last_became_active":"2024-05-27T00:39:21.588424+0000","last_became_peered":"2024-05-27T00:39:21.588424+0000","last_unstale":"2024-05-27T00:39:27.610077+0000","last_undegraded":"2024-05-27T00:39:27.610077+0000","last_fullsized":"2024-05-27T00:39:27.610077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:39:09.034246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133594+0000","last_change":"2024-05-27T00:39:21.587527+0000","last_active":"2024-05-27T00:39:28.133594+0000","last_peered":"2024-05-27T00:39:28.133594+0000","last_clean":"2024-05-27T00:39:28.133594+0000","last_became_active":"2024-05-27T00:39:21.587271+0000","last_became_peered":"2024-05-27T00:39:21.587271+0000","last_unstale":"2024-05-27T00:39:28.133594+0000","last_undegraded":"2024-05-27T00:39:28.133594+0000","last_fullsized":"2024-05-27T00:39:28.133594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610312+0000","last_change":"2024-05-27T00:39:21.600567+0000","last_active":"2024-05-27T00:39:27.610312+0000","last_peered":"2024-05-27T00:39:27.610312+0000","last_clean":"2024-05-27T00:39:27.610312+0000","last_became_active":"2024-05-27T00:39:21.600281+0000","last_became_peered":"2024-05-27T00:39:21.600281+0000","last_unstale":"2024-05-27T00:39:27.610312+0000","last_undegraded":"2024-05-27T00:39:27.610312+0000","last_fullsized":"2024-05-27T00:39:27.610312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604571+0000","last_change":"2024-05-27T00:39:21.591200+0000","last_active":"2024-05-27T00:39:27.604571+0000","last_peered":"2024-05-27T00:39:27.604571+0000","last_clean":"2024-05-27T00:39:27.604571+0000","last_became_active":"2024-05-27T00:39:21.590918+0000","last_became_peered":"2024-05-27T00:39:21.590918+0000","last_unstale":"2024-05-27T00:39:27.604571+0000","last_undegraded":"2024-05-27T00:39:27.604571+0000","last_fullsized":"2024-05-27T00:39:27.604571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835139+0000","last_change":"2024-05-27T00:39:21.593373+0000","last_active":"2024-05-27T00:39:28.835139+0000","last_peered":"2024-05-27T00:39:28.835139+0000","last_clean":"2024-05-27T00:39:28.835139+0000","last_became_active":"2024-05-27T00:39:21.590646+0000","last_became_peered":"2024-05-27T00:39:21.590646+0000","last_unstale":"2024-05-27T00:39:28.835139+0000","last_undegraded":"2024-05-27T00:39:28.835139+0000","last_fullsized":"2024-05-27T00:39:28.835139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610260+0000","last_change":"2024-05-27T00:39:21.600531+0000","last_active":"2024-05-27T00:39:27.610260+0000","last_peered":"2024-05-27T00:39:27.610260+0000","last_clean":"2024-05-27T00:39:27.610260+0000","last_became_active":"2024-05-27T00:39:21.600236+0000","last_became_peered":"2024-05-27T00:39:21.600236+0000","last_unstale":"2024-05-27T00:39:27.610260+0000","last_undegraded":"2024-05-27T00:39:27.610260+0000","last_fullsized":"2024-05-27T00:39:27.610260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609848+0000","last_change":"2024-05-27T00:39:21.588532+0000","last_active":"2024-05-27T00:39:27.609848+0000","last_peered":"2024-05-27T00:39:27.609848+0000","last_clean":"2024-05-27T00:39:27.609848+0000","last_became_active":"2024-05-27T00:39:21.587475+0000","last_became_peered":"2024-05-27T00:39:21.587475+0000","last_unstale":"2024-05-27T00:39:27.609848+0000","last_undegraded":"2024-05-27T00:39:27.609848+0000","last_fullsized":"2024-05-27T00:39:27.609848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:56.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609903+0000","last_change":"2024-05-27T00:39:21.601094+0000","last_active":"2024-05-27T00:39:27.609903+0000","last_peered":"2024-05-27T00:39:27.609903+0000","last_clean":"2024-05-27T00:39:27.609903+0000","last_became_active":"2024-05-27T00:39:21.600345+0000","last_became_peered":"2024-05-27T00:39:21.600345+0000","last_unstale":"2024-05-27T00:39:27.609903+0000","last_undegraded":"2024-05-27T00:39:27.609903+0000","last_fullsized":"2024-05-27T00:39:27.609903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:50.264233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609931+0000","last_change":"2024-05-27T00:39:21.583984+0000","last_active":"2024-05-27T00:39:27.609931+0000","last_peered":"2024-05-27T00:39:27.609931+0000","last_clean":"2024-05-27T00:39:27.609931+0000","last_became_active":"2024-05-27T00:39:21.583601+0000","last_became_peered":"2024-05-27T00:39:21.583601+0000","last_unstale":"2024-05-27T00:39:27.609931+0000","last_undegraded":"2024-05-27T00:39:27.609931+0000","last_fullsized":"2024-05-27T00:39:27.609931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:22.867237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133437+0000","last_change":"2024-05-27T00:39:21.583713+0000","last_active":"2024-05-27T00:39:28.133437+0000","last_peered":"2024-05-27T00:39:28.133437+0000","last_clean":"2024-05-27T00:39:28.133437+0000","last_became_active":"2024-05-27T00:39:21.583367+0000","last_became_peered":"2024-05-27T00:39:21.583367+0000","last_unstale":"2024-05-27T00:39:28.133437+0000","last_undegraded":"2024-05-27T00:39:28.133437+0000","last_fullsized":"2024-05-27T00:39:28.133437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834031+0000","last_change":"2024-05-27T00:39:21.597145+0000","last_active":"2024-05-27T00:39:28.834031+0000","last_peered":"2024-05-27T00:39:28.834031+0000","last_clean":"2024-05-27T00:39:28.834031+0000","last_became_active":"2024-05-27T00:39:21.596737+0000","last_became_peered":"2024-05-27T00:39:21.596737+0000","last_unstale":"2024-05-27T00:39:28.834031+0000","last_undegraded":"2024-05-27T00:39:28.834031+0000","last_fullsized":"2024-05-27T00:39:28.834031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:13:34.499546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834671+0000","last_change":"2024-05-27T00:39:21.586729+0000","last_active":"2024-05-27T00:39:28.834671+0000","last_peered":"2024-05-27T00:39:28.834671+0000","last_clean":"2024-05-27T00:39:28.834671+0000","last_became_active":"2024-05-27T00:39:21.586428+0000","last_became_peered":"2024-05-27T00:39:21.586428+0000","last_unstale":"2024-05-27T00:39:28.834671+0000","last_undegraded":"2024-05-27T00:39:28.834671+0000","last_fullsized":"2024-05-27T00:39:28.834671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:40.800479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832018+0000","last_change":"2024-05-27T00:39:21.591955+0000","last_active":"2024-05-27T00:39:28.832018+0000","last_peered":"2024-05-27T00:39:28.832018+0000","last_clean":"2024-05-27T00:39:28.832018+0000","last_became_active":"2024-05-27T00:39:21.591645+0000","last_became_peered":"2024-05-27T00:39:21.591645+0000","last_unstale":"2024-05-27T00:39:28.832018+0000","last_undegraded":"2024-05-27T00:39:28.832018+0000","last_fullsized":"2024-05-27T00:39:28.832018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610938+0000","last_change":"2024-05-27T00:39:21.582309+0000","last_active":"2024-05-27T00:39:27.610938+0000","last_peered":"2024-05-27T00:39:27.610938+0000","last_clean":"2024-05-27T00:39:27.610938+0000","last_became_active":"2024-05-27T00:39:21.582029+0000","last_became_peered":"2024-05-27T00:39:21.582029+0000","last_unstale":"2024-05-27T00:39:27.610938+0000","last_undegraded":"2024-05-27T00:39:27.610938+0000","last_fullsized":"2024-05-27T00:39:27.610938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609792+0000","last_change":"2024-05-27T00:39:21.583269+0000","last_active":"2024-05-27T00:39:27.609792+0000","last_peered":"2024-05-27T00:39:27.609792+0000","last_clean":"2024-05-27T00:39:27.609792+0000","last_became_active":"2024-05-27T00:39:21.582827+0000","last_became_peered":"2024-05-27T00:39:21.582827+0000","last_unstale":"2024-05-27T00:39:27.609792+0000","last_undegraded":"2024-05-27T00:39:27.609792+0000","last_fullsized":"2024-05-27T00:39:27.609792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604091+0000","last_change":"2024-05-27T00:39:21.600540+0000","last_active":"2024-05-27T00:39:27.604091+0000","last_peered":"2024-05-27T00:39:27.604091+0000","last_clean":"2024-05-27T00:39:27.604091+0000","last_became_active":"2024-05-27T00:39:21.600270+0000","last_became_peered":"2024-05-27T00:39:21.600270+0000","last_unstale":"2024-05-27T00:39:27.604091+0000","last_undegraded":"2024-05-27T00:39:27.604091+0000","last_fullsized":"2024-05-27T00:39:27.604091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133315+0000","last_change":"2024-05-27T00:39:21.587064+0000","last_active":"2024-05-27T00:39:28.133315+0000","last_peered":"2024-05-27T00:39:28.133315+0000","last_clean":"2024-05-27T00:39:28.133315+0000","last_became_active":"2024-05-27T00:39:21.586759+0000","last_became_peered":"2024-05-27T00:39:21.586759+0000","last_unstale":"2024-05-27T00:39:28.133315+0000","last_undegraded":"2024-05-27T00:39:28.133315+0000","last_fullsized":"2024-05-27T00:39:28.133315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:18.374156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609806+0000","last_change":"2024-05-27T00:39:21.594918+0000","last_active":"2024-05-27T00:39:27.609806+0000","last_peered":"2024-05-27T00:39:27.609806+0000","last_clean":"2024-05-27T00:39:27.609806+0000","last_became_active":"2024-05-27T00:39:21.594670+0000","last_became_peered":"2024-05-27T00:39:21.594670+0000","last_unstale":"2024-05-27T00:39:27.609806+0000","last_undegraded":"2024-05-27T00:39:27.609806+0000","last_fullsized":"2024-05-27T00:39:27.609806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:20:31.019485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834590+0000","last_change":"2024-05-27T00:39:21.600525+0000","last_active":"2024-05-27T00:39:28.834590+0000","last_peered":"2024-05-27T00:39:28.834590+0000","last_clean":"2024-05-27T00:39:28.834590+0000","last_became_active":"2024-05-27T00:39:21.600012+0000","last_became_peered":"2024-05-27T00:39:21.600012+0000","last_unstale":"2024-05-27T00:39:28.834590+0000","last_undegraded":"2024-05-27T00:39:28.834590+0000","last_fullsized":"2024-05-27T00:39:28.834590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:33.765408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.834173+0000","last_change":"2024-05-27T00:39:21.596158+0000","last_active":"2024-05-27T00:39:28.834173+0000","last_peered":"2024-05-27T00:39:28.834173+0000","last_clean":"2024-05-27T00:39:28.834173+0000","last_became_active":"2024-05-27T00:39:21.595655+0000","last_became_peered":"2024-05-27T00:39:21.595655+0000","last_unstale":"2024-05-27T00:39:28.834173+0000","last_undegraded":"2024-05-27T00:39:28.834173+0000","last_fullsized":"2024-05-27T00:39:28.834173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.835071+0000","last_change":"2024-05-27T00:39:21.594957+0000","last_active":"2024-05-27T00:39:28.835071+0000","last_peered":"2024-05-27T00:39:28.835071+0000","last_clean":"2024-05-27T00:39:28.835071+0000","last_became_active":"2024-05-27T00:39:21.594682+0000","last_became_peered":"2024-05-27T00:39:21.594682+0000","last_unstale":"2024-05-27T00:39:28.835071+0000","last_undegraded":"2024-05-27T00:39:28.835071+0000","last_fullsized":"2024-05-27T00:39:28.835071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833653+0000","last_change":"2024-05-27T00:39:21.589852+0000","last_active":"2024-05-27T00:39:28.833653+0000","last_peered":"2024-05-27T00:39:28.833653+0000","last_clean":"2024-05-27T00:39:28.833653+0000","last_became_active":"2024-05-27T00:39:21.589514+0000","last_became_peered":"2024-05-27T00:39:21.589514+0000","last_unstale":"2024-05-27T00:39:28.833653+0000","last_undegraded":"2024-05-27T00:39:28.833653+0000","last_fullsized":"2024-05-27T00:39:28.833653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:08.796084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609583+0000","last_change":"2024-05-27T00:39:21.586007+0000","last_active":"2024-05-27T00:39:27.609583+0000","last_peered":"2024-05-27T00:39:27.609583+0000","last_clean":"2024-05-27T00:39:27.609583+0000","last_became_active":"2024-05-27T00:39:21.585708+0000","last_became_peered":"2024-05-27T00:39:21.585708+0000","last_unstale":"2024-05-27T00:39:27.609583+0000","last_undegraded":"2024-05-27T00:39:27.609583+0000","last_fullsized":"2024-05-27T00:39:27.609583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:49.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.832658+0000","last_change":"2024-05-27T00:39:21.596377+0000","last_active":"2024-05-27T00:39:28.832658+0000","last_peered":"2024-05-27T00:39:28.832658+0000","last_clean":"2024-05-27T00:39:28.832658+0000","last_became_active":"2024-05-27T00:39:21.596055+0000","last_became_peered":"2024-05-27T00:39:21.596055+0000","last_unstale":"2024-05-27T00:39:28.832658+0000","last_undegraded":"2024-05-27T00:39:28.832658+0000","last_fullsized":"2024-05-27T00:39:28.832658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:46:24.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.609929+0000","last_change":"2024-05-27T00:39:21.583846+0000","last_active":"2024-05-27T00:39:27.609929+0000","last_peered":"2024-05-27T00:39:27.609929+0000","last_clean":"2024-05-27T00:39:27.609929+0000","last_became_active":"2024-05-27T00:39:21.583284+0000","last_became_peered":"2024-05-27T00:39:21.583284+0000","last_unstale":"2024-05-27T00:39:27.609929+0000","last_undegraded":"2024-05-27T00:39:27.609929+0000","last_fullsized":"2024-05-27T00:39:27.609929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:10:08.620655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833536+0000","last_change":"2024-05-27T00:39:21.586873+0000","last_active":"2024-05-27T00:39:28.833536+0000","last_peered":"2024-05-27T00:39:28.833536+0000","last_clean":"2024-05-27T00:39:28.833536+0000","last_became_active":"2024-05-27T00:39:21.586593+0000","last_became_peered":"2024-05-27T00:39:21.586593+0000","last_unstale":"2024-05-27T00:39:28.833536+0000","last_undegraded":"2024-05-27T00:39:28.833536+0000","last_fullsized":"2024-05-27T00:39:28.833536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:43.541844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.610399+0000","last_change":"2024-05-27T00:39:21.592963+0000","last_active":"2024-05-27T00:39:27.610399+0000","last_peered":"2024-05-27T00:39:27.610399+0000","last_clean":"2024-05-27T00:39:27.610399+0000","last_became_active":"2024-05-27T00:39:21.588049+0000","last_became_peered":"2024-05-27T00:39:21.588049+0000","last_unstale":"2024-05-27T00:39:27.610399+0000","last_undegraded":"2024-05-27T00:39:27.610399+0000","last_fullsized":"2024-05-27T00:39:27.610399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:38.702869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.133490+0000","last_change":"2024-05-27T00:39:21.588239+0000","last_active":"2024-05-27T00:39:28.133490+0000","last_peered":"2024-05-27T00:39:28.133490+0000","last_clean":"2024-05-27T00:39:28.133490+0000","last_became_active":"2024-05-27T00:39:21.587936+0000","last_became_peered":"2024-05-27T00:39:21.587936+0000","last_unstale":"2024-05-27T00:39:28.133490+0000","last_undegraded":"2024-05-27T00:39:28.133490+0000","last_fullsized":"2024-05-27T00:39:28.133490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:42.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:27.604778+0000","last_change":"2024-05-27T00:39:21.591553+0000","last_active":"2024-05-27T00:39:27.604778+0000","last_peered":"2024-05-27T00:39:27.604778+0000","last_clean":"2024-05-27T00:39:27.604778+0000","last_became_active":"2024-05-27T00:39:21.591286+0000","last_became_peered":"2024-05-27T00:39:21.591286+0000","last_unstale":"2024-05-27T00:39:27.604778+0000","last_undegraded":"2024-05-27T00:39:27.604778+0000","last_fullsized":"2024-05-27T00:39:27.604778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:17:16.530124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-27T00:39:28.833459+0000","last_change":"2024-05-27T00:39:21.591695+0000","last_active":"2024-05-27T00:39:28.833459+0000","last_peered":"2024-05-27T00:39:28.833459+0000","last_clean":"2024-05-27T00:39:28.833459+0000","last_became_active":"2024-05-27T00:39:21.591384+0000","last_became_peered":"2024-05-27T00:39:21.591384+0000","last_unstale":"2024-05-27T00:39:28.833459+0000","last_undegraded":"2024-05-27T00:39:28.833459+0000","last_fullsized":"2024-05-27T00:39:28.833459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:20.551903+0000","last_clean_scrub_stamp":"2024-05-27T00:39:20.551903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:44.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:39:45.967 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-27T00:39:45.968 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T00:39:45.968 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-27T00:39:45.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.098+0000 7f1db7c8c640 1 Processor -- start 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.099+0000 7f1db7c8c640 1 -- start start 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.099+0000 7f1db7c8c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1db0144000 0x7f1db00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.099+0000 7f1db7c8c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1db006b390 0x7f1db00cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.099+0000 7f1db7c8c640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db00b3440 con 0x7f1db00cdc80 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.099+0000 7f1db7c8c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1db00b42f0 con 0x7f1db0144000 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.099+0000 7f1db7c8c640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1db00b36e0 con 0x7f1db006b390 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1db0144000 0x7f1db00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1db0144000 0x7f1db00b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51284/0 (socket says 172.21.15.111:51284) 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 -- 172.21.15.111:0/2835653089 learned_addr learned my addr 172.21.15.111:0/2835653089 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5227640 1 --2- 172.21.15.111:0/2835653089 >> v2:172.21.15.153:3300/0 conn(0x7f1db006b390 0x7f1db00cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 -- 172.21.15.111:0/2835653089 >> v2:172.21.15.153:3300/0 conn(0x7f1db006b390 msgr2=0x7f1db00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 --2- 172.21.15.111:0/2835653089 >> v2:172.21.15.153:3300/0 conn(0x7f1db006b390 0x7f1db00cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 -- 172.21.15.111:0/2835653089 >> v1:172.21.15.153:6789/0 conn(0x7f1db00cdc80 legacy=0x7f1db00726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.100+0000 7f1db5a28640 1 -- 172.21.15.111:0/2835653089 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db00b39c0 con 0x7f1db0144000 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db5a28640 1 --2- 172.21.15.111:0/2835653089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db0144000 0x7f1db00b2f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1db0072650 tx=0x7f1da402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e6f9c31e94b872 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db4a26640 1 -- 172.21.15.111:0/2835653089 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40325a0 con 0x7f1db0144000 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db4a26640 1 -- 172.21.15.111:0/2835653089 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da4038070 con 0x7f1db0144000 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db4a26640 1 -- 172.21.15.111:0/2835653089 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4032960 con 0x7f1db0144000 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 -- 172.21.15.111:0/2835653089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db0144000 msgr2=0x7f1db00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/2835653089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db0144000 0x7f1db00b2f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1db0072650 tx=0x7f1da402c8e0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 -- 172.21.15.111:0/2835653089 shutdown_connections 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/2835653089 >> v2:172.21.15.153:3300/0 conn(0x7f1db006b390 0x7f1db00cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/2835653089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db0144000 0x7f1db00b2f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 -- 172.21.15.111:0/2835653089 >> 172.21.15.111:0/2835653089 conn(0x7f1db00b1c00 msgr2=0x7f1db00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 -- 172.21.15.111:0/2835653089 shutdown_connections 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.101+0000 7f1db7c8c640 1 -- 172.21.15.111:0/2835653089 wait complete. 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.102+0000 7f1db7c8c640 1 Processor -- start 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 -- start start 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 0x7f1db013c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1db00cdc80 0x7f1db006b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1db0144000 0x7f1db006bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0072de0 con 0x7f1db006b390 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1db00b3f80 con 0x7f1db00cdc80 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db7c8c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1db0072f60 con 0x7f1db0144000 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db5a28640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 0x7f1db013c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db5a28640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 0x7f1db013c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51288/0 (socket says 172.21.15.111:51288) 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.103+0000 7f1db5a28640 1 -- 172.21.15.111:0/3464119614 learned_addr learned my addr 172.21.15.111:0/3464119614 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5a28640 1 -- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1db0144000 msgr2=0x7f1db006bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db6229640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1db0144000 0x7f1db006bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5227640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1db00cdc80 0x7f1db006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5a28640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1db0144000 0x7f1db006bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5a28640 1 -- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1db00cdc80 msgr2=0x7f1db006b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5a28640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1db00cdc80 0x7f1db006b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5a28640 1 -- 172.21.15.111:0/3464119614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db01f0cf0 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5227640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1db00cdc80 0x7f1db006b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db6229640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1db0144000 0x7f1db006bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db5a28640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 0x7f1db013c820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b2c0 tx=0x7f1da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4033840 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da4033a00 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db0147650 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4033df0 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.104+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db0147c10 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.105+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db0068680 con 0x7f1db006b390 2024-05-27T00:39:46.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.105+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1da404d020 con 0x7f1db006b390 2024-05-27T00:39:46.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.106+0000 7f1da27fc640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d9406be30 0x7f1d9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.106+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1da4030020 con 0x7f1db006b390 2024-05-27T00:39:46.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.106+0000 7f1db5227640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d9406be30 0x7f1d9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.107+0000 7f1db5227640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d9406be30 0x7f1d9406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1dac006fd0 tx=0x7f1dac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.110+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1da4041070 con 0x7f1db006b390 2024-05-27T00:39:46.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.321+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1db00723a0 con 0x7f1db006b390 2024-05-27T00:39:46.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.321+0000 7f1da27fc640 1 -- 172.21.15.111:0/3464119614 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1da407c630 con 0x7f1db006b390 2024-05-27T00:39:46.323 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:46.323 INFO:teuthology.orchestra.run.smithi111.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T00:39:46.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d9406be30 msgr2=0x7f1d9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d9406be30 0x7f1d9406e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1dac006fd0 tx=0x7f1dac008040 comp rx=0 tx=0).stop 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 msgr2=0x7f1db013c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 0x7f1db013c820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b2c0 tx=0x7f1da40046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 shutdown_connections 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d9406be30 0x7f1d9406e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1db0144000 0x7f1db006bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1db00cdc80 0x7f1db006b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 --2- 172.21.15.111:0/3464119614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1db006b390 0x7f1db013c820 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 >> 172.21.15.111:0/3464119614 conn(0x7f1db00b1c00 msgr2=0x7f1db00af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 shutdown_connections 2024-05-27T00:39:46.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.323+0000 7f1db7c8c640 1 -- 172.21.15.111:0/3464119614 wait complete. 2024-05-27T00:39:46.335 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-27T00:39:46.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:46.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.433+0000 7f85d79ce640 1 Processor -- start 2024-05-27T00:39:46.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d79ce640 1 -- start start 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d79ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85d0161e50 0x7f85d01622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d79ce640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00d2c10 0x7f85d00d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d79ce640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d006cc70 con 0x7f85d00b3630 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d79ce640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f85d00d35e0 con 0x7f85d0161e50 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d79ce640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f85d00d1e50 con 0x7f85d00d2c10 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.434+0000 7f85d4f69640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85d0161e50 0x7f85d01622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d576a640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f85d00b3630 0x7f85d00b72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36748/0 (socket says 172.21.15.111:36748) 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d4f69640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f85d0161e50 0x7f85d01622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51300/0 (socket says 172.21.15.111:51300) 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d576a640 1 -- 172.21.15.111:0/1135840751 learned_addr learned my addr 172.21.15.111:0/1135840751 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d5f6b640 1 --2- 172.21.15.111:0/1135840751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00d2c10 0x7f85d00d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d4f69640 1 -- 172.21.15.111:0/1135840751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00d2c10 msgr2=0x7f85d00d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:46.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d4f69640 1 --2- 172.21.15.111:0/1135840751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00d2c10 0x7f85d00d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d4f69640 1 -- 172.21.15.111:0/1135840751 >> v1:172.21.15.111:6789/0 conn(0x7f85d00b3630 legacy=0x7f85d00b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d4f69640 1 -- 172.21.15.111:0/1135840751 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d00d2130 con 0x7f85d0161e50 2024-05-27T00:39:46.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85cffff640 1 -- 172.21.15.111:0/1135840751 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131760141 0 0) 0x7f85d006cc70 con 0x7f85d00b3630 2024-05-27T00:39:46.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.435+0000 7f85d4f69640 1 --2- 172.21.15.111:0/1135840751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d01622e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f85c40097a0 tx=0x7f85c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940ecaee5e242bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85cffff640 1 -- 172.21.15.111:0/1135840751 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4037070 con 0x7f85d0161e50 2024-05-27T00:39:46.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85cffff640 1 -- 172.21.15.111:0/1135840751 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c4032440 con 0x7f85d0161e50 2024-05-27T00:39:46.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85cffff640 1 -- 172.21.15.111:0/1135840751 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4032810 con 0x7f85d0161e50 2024-05-27T00:39:46.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 -- 172.21.15.111:0/1135840751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 msgr2=0x7f85d01622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1135840751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d01622e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f85c40097a0 tx=0x7f85c402a180 comp rx=0 tx=0).stop 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 -- 172.21.15.111:0/1135840751 shutdown_connections 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1135840751 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00d2c10 0x7f85d00d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1135840751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d01622e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 -- 172.21.15.111:0/1135840751 >> 172.21.15.111:0/1135840751 conn(0x7f85d00d42c0 msgr2=0x7f85d00d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 -- 172.21.15.111:0/1135840751 shutdown_connections 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.436+0000 7f85d79ce640 1 -- 172.21.15.111:0/1135840751 wait complete. 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.437+0000 7f85d79ce640 1 Processor -- start 2024-05-27T00:39:46.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.437+0000 7f85d79ce640 1 -- start start 2024-05-27T00:39:46.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d79ce640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00b3630 0x7f85d00ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d79ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d00d2c10 0x7f85d0152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d79ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d79ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85d00b79b0 con 0x7f85d0161e50 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d79ce640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f85d00d26f0 con 0x7f85d00d2c10 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d79ce640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f85d006bfb0 con 0x7f85d00b3630 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d5f6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d5f6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d00cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51314/0 (socket says 172.21.15.111:51314) 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.438+0000 7f85d5f6b640 1 -- 172.21.15.111:0/1859716947 learned_addr learned my addr 172.21.15.111:0/1859716947 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d5f6b640 1 -- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00b3630 msgr2=0x7f85d00ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d5f6b640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00b3630 0x7f85d00ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d4f69640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d00d2c10 0x7f85d0152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d5f6b640 1 -- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d00d2c10 msgr2=0x7f85d0152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d5f6b640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d00d2c10 0x7f85d0152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d5f6b640 1 -- 172.21.15.111:0/1859716947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d00ca0e0 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d5f6b640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d00cb1b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85c800c7f0 tx=0x7f85c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8015070 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8004220 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d01f43d0 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8007620 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.439+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01f4990 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.440+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0068680 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.440+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85c80079d0 con 0x7f85d0161e50 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.441+0000 7f85cdffb640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85bc06be30 0x7f85bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.441+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f85c808b220 con 0x7f85d0161e50 2024-05-27T00:39:46.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.441+0000 7f85d576a640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85bc06be30 0x7f85bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.444 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.442+0000 7f85d576a640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85bc06be30 0x7f85bc06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85c0001560 tx=0x7f85c0007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.445+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f85c8059890 con 0x7f85d0161e50 2024-05-27T00:39:46.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.618+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85d01f4d30 con 0x7f85d0161e50 2024-05-27T00:39:46.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.619+0000 7f85cdffb640 1 -- 172.21.15.111:0/1859716947 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f85c8059190 con 0x7f85d0161e50 2024-05-27T00:39:46.621 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:46.621 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:46.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85bc06be30 msgr2=0x7f85bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85bc06be30 0x7f85bc06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85c0001560 tx=0x7f85c0007ca0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 msgr2=0x7f85d00cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d00cb1b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85c800c7f0 tx=0x7f85c800ccb0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 shutdown_connections 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85bc06be30 0x7f85bc06e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0161e50 0x7f85d00cb1b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85d00d2c10 0x7f85d0152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 --2- 172.21.15.111:0/1859716947 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f85d00b3630 0x7f85d00ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 >> 172.21.15.111:0/1859716947 conn(0x7f85d00d42c0 msgr2=0x7f85d0161560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 shutdown_connections 2024-05-27T00:39:46.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.621+0000 7f85d79ce640 1 -- 172.21.15.111:0/1859716947 wait complete. 2024-05-27T00:39:46.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:39:46.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:46.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.735+0000 7ffbbbeae640 1 Processor -- start 2024-05-27T00:39:46.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.735+0000 7ffbbbeae640 1 -- start start 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbbbeae640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffbb406b390 0x7ffbb40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbbbeae640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffbb40cdc80 0x7ffbb40726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbbbeae640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40b36e0 con 0x7ffbb4144000 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbbbeae640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b42f0 con 0x7ffbb406b390 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbbbeae640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b39c0 con 0x7ffbb40cdc80 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffbb406b390 0x7ffbb40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ffbb406b390 0x7ffbb40cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51324/0 (socket says 172.21.15.111:51324) 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 -- 172.21.15.111:0/1688552282 learned_addr learned my addr 172.21.15.111:0/1688552282 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9c4a640 1 --1- 172.21.15.111:0/1688552282 >> v1:172.21.15.111:6789/0 conn(0x7ffbb4144000 0x7ffbb40b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36762/0 (socket says 172.21.15.111:36762) 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbba44b640 1 --2- 172.21.15.111:0/1688552282 >> v2:172.21.15.153:3300/0 conn(0x7ffbb40cdc80 0x7ffbb40726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 -- 172.21.15.111:0/1688552282 >> v2:172.21.15.153:3300/0 conn(0x7ffbb40cdc80 msgr2=0x7ffbb40726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 --2- 172.21.15.111:0/1688552282 >> v2:172.21.15.153:3300/0 conn(0x7ffbb40cdc80 0x7ffbb40726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb8c48640 1 -- 172.21.15.111:0/1688552282 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942197867 0 0) 0x7ffbb40b36e0 con 0x7ffbb4144000 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 -- 172.21.15.111:0/1688552282 >> v1:172.21.15.111:6789/0 conn(0x7ffbb4144000 legacy=0x7ffbb40b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbb9449640 1 -- 172.21.15.111:0/1688552282 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b3ca0 con 0x7ffbb406b390 2024-05-27T00:39:46.740 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.736+0000 7ffbba44b640 1 --2- 172.21.15.111:0/1688552282 >> v2:172.21.15.153:3300/0 conn(0x7ffbb40cdc80 0x7ffbb40726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.737+0000 7ffbb9449640 1 --2- 172.21.15.111:0/1688552282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb406b390 0x7ffbb40cd740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffbb406e340 tx=0x7ffba802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475908f6af540a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.737+0000 7ffbb8c48640 1 -- 172.21.15.111:0/1688552282 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8037070 con 0x7ffbb406b390 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.737+0000 7ffbb8c48640 1 -- 172.21.15.111:0/1688552282 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba8032440 con 0x7ffbb406b390 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.737+0000 7ffbb8c48640 1 -- 172.21.15.111:0/1688552282 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8032810 con 0x7ffbb406b390 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1688552282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb406b390 msgr2=0x7ffbb40cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1688552282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb406b390 0x7ffbb40cd740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffbb406e340 tx=0x7ffba802a270 comp rx=0 tx=0).stop 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1688552282 shutdown_connections 2024-05-27T00:39:46.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1688552282 >> v2:172.21.15.153:3300/0 conn(0x7ffbb40cdc80 0x7ffbb40726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1688552282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb406b390 0x7ffbb40cd740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1688552282 >> 172.21.15.111:0/1688552282 conn(0x7ffbb40b1c00 msgr2=0x7ffbb40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:46.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1688552282 shutdown_connections 2024-05-27T00:39:46.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.738+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1688552282 wait complete. 2024-05-27T00:39:46.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.739+0000 7ffbbbeae640 1 Processor -- start 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.739+0000 7ffbbbeae640 1 -- start start 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbbbeae640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb406b390 0x7ffbb40a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbbbeae640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb40cdc80 0x7ffbb41591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbbbeae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 0x7ffbb413ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbbbeae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4072d90 con 0x7ffbb4144000 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbbbeae640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4072c10 con 0x7ffbb406b390 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbbbeae640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb4072f10 con 0x7ffbb40cdc80 2024-05-27T00:39:46.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 0x7ffbb413ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 0x7ffbb413ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51338/0 (socket says 172.21.15.111:51338) 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 -- 172.21.15.111:0/1085110819 learned_addr learned my addr 172.21.15.111:0/1085110819 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbb9449640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb40cdc80 0x7ffbb41591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbb9c4a640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb406b390 0x7ffbb40a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 -- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb40cdc80 msgr2=0x7ffbb41591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb40cdc80 0x7ffbb41591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 -- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb406b390 msgr2=0x7ffbb40a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb406b390 0x7ffbb40a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 -- 172.21.15.111:0/1085110819 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb4138570 con 0x7ffbb4144000 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.740+0000 7ffbba44b640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 0x7ffbb413ac10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c7f0 tx=0x7ffbac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.741+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac015070 con 0x7ffbb4144000 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.741+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac0077c0 con 0x7ffbb4144000 2024-05-27T00:39:46.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.741+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb4138850 con 0x7ffbb4144000 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.741+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007bf0 con 0x7ffbb4144000 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.741+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb41390f0 con 0x7ffbb4144000 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.742+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4068680 con 0x7ffbb4144000 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.742+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbac018070 con 0x7ffbb4144000 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.742+0000 7ffbb27fc640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ffba006be30 0x7ffba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.743+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffbac08b040 con 0x7ffbb4144000 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.743+0000 7ffbb9c4a640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ffba006be30 0x7ffba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:46.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.743+0000 7ffbb9c4a640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ffba006be30 0x7ffba006e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffba40012e0 tx=0x7ffba4001630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:46.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.747+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffbac059060 con 0x7ffbb4144000 2024-05-27T00:39:46.926 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.925+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbb40b2610 con 0x7ffbb4144000 2024-05-27T00:39:46.926 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.925+0000 7ffbb27fc640 1 -- 172.21.15.111:0/1085110819 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7ffbac05d360 con 0x7ffbb4144000 2024-05-27T00:39:46.926 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:46.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ffba006be30 msgr2=0x7ffba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ffba006be30 0x7ffba006e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffba40012e0 tx=0x7ffba4001630 comp rx=0 tx=0).stop 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 msgr2=0x7ffbb413ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 0x7ffbb413ac10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c7f0 tx=0x7ffbac00ccb0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 shutdown_connections 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ffba006be30 0x7ffba006e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb4144000 0x7ffbb413ac10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ffbb40cdc80 0x7ffbb41591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 --2- 172.21.15.111:0/1085110819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb406b390 0x7ffbb40a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 >> 172.21.15.111:0/1085110819 conn(0x7ffbb40b1c00 msgr2=0x7ffbb40cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 shutdown_connections 2024-05-27T00:39:46.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:46.927+0000 7ffbbbeae640 1 -- 172.21.15.111:0/1085110819 wait complete. 2024-05-27T00:39:46.939 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:39:47.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.041+0000 7f1e4d2ee640 1 Processor -- start 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.041+0000 7f1e4d2ee640 1 -- start start 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e4d2ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e4815c0b0 0x7f1e480d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e4d2ee640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1e480cf020 0x7f1e4815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e4d2ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e480b4e40 con 0x7f1e480ce0e0 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e4d2ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1e480ccbf0 con 0x7f1e4815c0b0 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e4d2ee640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1e48070fe0 con 0x7f1e480cf020 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e46ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e4815c0b0 0x7f1e480d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e46ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1e4815c0b0 0x7f1e480d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51346/0 (socket says 172.21.15.111:51346) 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e467fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1e480ce0e0 0x7f1e480ce910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36768/0 (socket says 172.21.15.111:36768) 2024-05-27T00:39:47.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.042+0000 7f1e46ffd640 1 -- 172.21.15.111:0/2741621781 learned_addr learned my addr 172.21.15.111:0/2741621781 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e46ffd640 1 -- 172.21.15.111:0/2741621781 >> v2:172.21.15.153:3300/0 conn(0x7f1e480cf020 msgr2=0x7f1e4815fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e46ffd640 1 --2- 172.21.15.111:0/2741621781 >> v2:172.21.15.153:3300/0 conn(0x7f1e480cf020 0x7f1e4815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e46ffd640 1 -- 172.21.15.111:0/2741621781 >> v1:172.21.15.111:6789/0 conn(0x7f1e480ce0e0 legacy=0x7f1e480ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e45ffb640 1 -- 172.21.15.111:0/2741621781 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979566319 0 0) 0x7f1e480b4e40 con 0x7f1e480ce0e0 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e46ffd640 1 -- 172.21.15.111:0/2741621781 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e48070a40 con 0x7f1e4815c0b0 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e46ffd640 1 --2- 172.21.15.111:0/2741621781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e4815c0b0 0x7f1e480d1920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e480c8f20 tx=0x7f1e3802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee8492f6c9929a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.043+0000 7f1e45ffb640 1 -- 172.21.15.111:0/2741621781 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38036070 con 0x7f1e4815c0b0 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e45ffb640 1 -- 172.21.15.111:0/2741621781 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38031600 con 0x7f1e4815c0b0 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e45ffb640 1 -- 172.21.15.111:0/2741621781 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380319d0 con 0x7f1e4815c0b0 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/2741621781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e4815c0b0 msgr2=0x7f1e480d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/2741621781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e4815c0b0 0x7f1e480d1920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e480c8f20 tx=0x7f1e3802a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/2741621781 shutdown_connections 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/2741621781 >> v2:172.21.15.153:3300/0 conn(0x7f1e480cf020 0x7f1e4815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/2741621781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e4815c0b0 0x7f1e480d1920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/2741621781 >> 172.21.15.111:0/2741621781 conn(0x7f1e4806ba70 msgr2=0x7f1e480d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/2741621781 shutdown_connections 2024-05-27T00:39:47.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.044+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/2741621781 wait complete. 2024-05-27T00:39:47.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.045+0000 7f1e4d2ee640 1 Processor -- start 2024-05-27T00:39:47.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.045+0000 7f1e4d2ee640 1 -- start start 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e4d2ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 0x7f1e48143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e4d2ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e480cf020 0x7f1e480cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e4d2ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4815c0b0 0x7f1e480cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e4d2ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480cc6d0 con 0x7f1e480ce0e0 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e4d2ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48070d00 con 0x7f1e4815c0b0 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e4d2ee640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4815c490 con 0x7f1e480cf020 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 0x7f1e48143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 0x7f1e48143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51350/0 (socket says 172.21.15.111:51350) 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 -- 172.21.15.111:0/1890190576 learned_addr learned my addr 172.21.15.111:0/1890190576 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 -- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e480cf020 msgr2=0x7f1e480cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e480cf020 0x7f1e480cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 -- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4815c0b0 msgr2=0x7f1e480cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e477fe640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4815c0b0 0x7f1e480cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4815c0b0 0x7f1e480cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e46ffd640 1 -- 172.21.15.111:0/1890190576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e48140180 con 0x7f1e480ce0e0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.046+0000 7f1e477fe640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4815c0b0 0x7f1e480cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.047+0000 7f1e46ffd640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 0x7f1e48143670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e38034040 tx=0x7f1e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.047+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38041020 con 0x7f1e480ce0e0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.047+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38007800 con 0x7f1e480ce0e0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.047+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e48140460 con 0x7f1e480ce0e0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.047+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38036040 con 0x7f1e480ce0e0 2024-05-27T00:39:47.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.047+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e48140a20 con 0x7f1e480ce0e0 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.048+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48068680 con 0x7f1e480ce0e0 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.048+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e38044020 con 0x7f1e480ce0e0 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.049+0000 7f1e3f7fe640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1e2806be30 0x7f1e2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.049+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1e380abfa0 con 0x7f1e480ce0e0 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.049+0000 7f1e467fc640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1e2806be30 0x7f1e2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.050+0000 7f1e467fc640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1e2806be30 0x7f1e2806e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e30007fd0 tx=0x7f1e30007900 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.053+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1e3804b030 con 0x7f1e480ce0e0 2024-05-27T00:39:47.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.226+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e4815fa30 con 0x7f1e480ce0e0 2024-05-27T00:39:47.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.226+0000 7f1e3f7fe640 1 -- 172.21.15.111:0/1890190576 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f1e3807a610 con 0x7f1e480ce0e0 2024-05-27T00:39:47.228 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1e2806be30 msgr2=0x7f1e2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1e2806be30 0x7f1e2806e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e30007fd0 tx=0x7f1e30007900 comp rx=0 tx=0).stop 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 msgr2=0x7f1e48143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 0x7f1e48143670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e38034040 tx=0x7f1e38004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 shutdown_connections 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1e2806be30 0x7f1e2806e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4815c0b0 0x7f1e480cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1e480cf020 0x7f1e480cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 --2- 172.21.15.111:0/1890190576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e480ce0e0 0x7f1e48143670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 >> 172.21.15.111:0/1890190576 conn(0x7f1e4806ba70 msgr2=0x7f1e4815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 shutdown_connections 2024-05-27T00:39:47.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.228+0000 7f1e4d2ee640 1 -- 172.21.15.111:0/1890190576 wait complete. 2024-05-27T00:39:47.241 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.343+0000 7fbb6a9f4640 1 Processor -- start 2024-05-27T00:39:47.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.343+0000 7fbb6a9f4640 1 -- start start 2024-05-27T00:39:47.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb6a9f4640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbb64144000 0x7fbb640b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb6a9f4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbb640726d0 0x7fbb64072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb6a9f4640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb64073020 con 0x7fbb6406b390 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb6a9f4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fbb640b42f0 con 0x7fbb640726d0 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb6a9f4640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbb640b3440 con 0x7fbb64144000 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbb640726d0 0x7fbb64072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbb640726d0 0x7fbb64072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51366/0 (socket says 172.21.15.111:51366) 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 -- 172.21.15.111:0/4085034994 learned_addr learned my addr 172.21.15.111:0/4085034994 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb63fff640 1 --2- 172.21.15.111:0/4085034994 >> v2:172.21.15.153:3300/0 conn(0x7fbb64144000 0x7fbb640b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 -- 172.21.15.111:0/4085034994 >> v2:172.21.15.153:3300/0 conn(0x7fbb64144000 msgr2=0x7fbb640b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 --2- 172.21.15.111:0/4085034994 >> v2:172.21.15.153:3300/0 conn(0x7fbb64144000 0x7fbb640b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 -- 172.21.15.111:0/4085034994 >> v1:172.21.15.153:6789/0 conn(0x7fbb6406b390 legacy=0x7fbb640cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb68f91640 1 -- 172.21.15.111:0/4085034994 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb640b36e0 con 0x7fbb640726d0 2024-05-27T00:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.344+0000 7fbb63fff640 1 --2- 172.21.15.111:0/4085034994 >> v2:172.21.15.153:3300/0 conn(0x7fbb64144000 0x7fbb640b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb68f91640 1 --2- 172.21.15.111:0/4085034994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb640726d0 0x7fbb64072ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb640726a0 tx=0x7fbb5c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242353d3861c1e01 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb62ffd640 1 -- 172.21.15.111:0/4085034994 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c036070 con 0x7fbb640726d0 2024-05-27T00:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb62ffd640 1 -- 172.21.15.111:0/4085034994 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c031600 con 0x7fbb640726d0 2024-05-27T00:39:47.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb62ffd640 1 -- 172.21.15.111:0/4085034994 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0319d0 con 0x7fbb640726d0 2024-05-27T00:39:47.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/4085034994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb640726d0 msgr2=0x7fbb64072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/4085034994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb640726d0 0x7fbb64072ae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb640726a0 tx=0x7fbb5c02a730 comp rx=0 tx=0).stop 2024-05-27T00:39:47.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/4085034994 shutdown_connections 2024-05-27T00:39:47.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/4085034994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb640726d0 0x7fbb64072ae0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/4085034994 >> v2:172.21.15.153:3300/0 conn(0x7fbb64144000 0x7fbb640b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/4085034994 >> 172.21.15.111:0/4085034994 conn(0x7fbb640b1c00 msgr2=0x7fbb640b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/4085034994 shutdown_connections 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.345+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/4085034994 wait complete. 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 Processor -- start 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 -- start start 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 0x7fbb641f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb640726d0 0x7fbb641f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb64144000 0x7fbb641f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbb640b3f80 con 0x7fbb6406b390 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbb640b3ca0 con 0x7fbb64144000 2024-05-27T00:39:47.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.347+0000 7fbb6a9f4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbb640cde50 con 0x7fbb640726d0 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 0x7fbb641f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 0x7fbb641f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51378/0 (socket says 172.21.15.111:51378) 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 -- 172.21.15.111:0/1042121831 learned_addr learned my addr 172.21.15.111:0/1042121831 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb68f91640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb64144000 0x7fbb641f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb637fe640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb640726d0 0x7fbb641f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 -- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb640726d0 msgr2=0x7fbb641f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb640726d0 0x7fbb641f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 -- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb64144000 msgr2=0x7fbb641f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb64144000 0x7fbb641f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 -- 172.21.15.111:0/1042121831 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641f2640 con 0x7fbb6406b390 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb63fff640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 0x7fbb641f5140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400e7e0 tx=0x7fbb5400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400f040 con 0x7fbb6406b390 2024-05-27T00:39:47.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb54004030 con 0x7fbb6406b390 2024-05-27T00:39:47.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb640c50c0 con 0x7fbb6406b390 2024-05-27T00:39:47.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54004400 con 0x7fbb6406b390 2024-05-27T00:39:47.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.348+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb640c5680 con 0x7fbb6406b390 2024-05-27T00:39:47.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.349+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb64068680 con 0x7fbb6406b390 2024-05-27T00:39:47.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.350+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb5400c6d0 con 0x7fbb6406b390 2024-05-27T00:39:47.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.350+0000 7fbb60ff9640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbb4406be30 0x7fbb4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.350+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb5408ae20 con 0x7fbb6406b390 2024-05-27T00:39:47.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.350+0000 7fbb637fe640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbb4406be30 0x7fbb4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.351+0000 7fbb637fe640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbb4406be30 0x7fbb4406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c007fd0 tx=0x7fbb4c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.354+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb54017020 con 0x7fbb6406b390 2024-05-27T00:39:47.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.529+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb640af360 con 0x7fbb6406b390 2024-05-27T00:39:47.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.529+0000 7fbb60ff9640 1 -- 172.21.15.111:0/1042121831 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fbb54058ed0 con 0x7fbb6406b390 2024-05-27T00:39:47.530 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:39:47.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbb4406be30 msgr2=0x7fbb4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbb4406be30 0x7fbb4406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c007fd0 tx=0x7fbb4c009040 comp rx=0 tx=0).stop 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 msgr2=0x7fbb641f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 0x7fbb641f5140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400e7e0 tx=0x7fbb5400eca0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 shutdown_connections 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbb4406be30 0x7fbb4406e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb64144000 0x7fbb641f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbb640726d0 0x7fbb641f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 --2- 172.21.15.111:0/1042121831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb6406b390 0x7fbb641f5140 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 >> 172.21.15.111:0/1042121831 conn(0x7fbb640b1c00 msgr2=0x7fbb640cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 shutdown_connections 2024-05-27T00:39:47.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.532+0000 7fbb6a9f4640 1 -- 172.21.15.111:0/1042121831 wait complete. 2024-05-27T00:39:47.543 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:39:47.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.645+0000 7f1432412640 1 Processor -- start 2024-05-27T00:39:47.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.645+0000 7f1432412640 1 -- start start 2024-05-27T00:39:47.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.645+0000 7f1432412640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f142c144000 0x7f142c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.645+0000 7f1432412640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f142c06b390 0x7f142c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f1432412640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f142c0b3440 con 0x7f142c0cdc80 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f1432412640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f142c0b42f0 con 0x7f142c06b390 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f1432412640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f142c0b36e0 con 0x7f142c144000 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f142c06b390 0x7f142c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f142c06b390 0x7f142c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51394/0 (socket says 172.21.15.111:51394) 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f14309af640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f142c0cdc80 0x7f142c0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36778/0 (socket says 172.21.15.111:36778) 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 -- 172.21.15.111:0/2399303561 learned_addr learned my addr 172.21.15.111:0/2399303561 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142bfff640 1 --2- 172.21.15.111:0/2399303561 >> v2:172.21.15.153:3300/0 conn(0x7f142c144000 0x7f142c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 -- 172.21.15.111:0/2399303561 >> v2:172.21.15.153:3300/0 conn(0x7f142c144000 msgr2=0x7f142c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 --2- 172.21.15.111:0/2399303561 >> v2:172.21.15.153:3300/0 conn(0x7f142c144000 0x7f142c0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 -- 172.21.15.111:0/2399303561 >> v1:172.21.15.111:6789/0 conn(0x7f142c0cdc80 legacy=0x7f142c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142b7fe640 1 -- 172.21.15.111:0/2399303561 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c0b39c0 con 0x7f142c06b390 2024-05-27T00:39:47.649 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142affd640 1 -- 172.21.15.111:0/2399303561 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173082567 0 0) 0x7f142c0b3440 con 0x7f142c0cdc80 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.646+0000 7f142bfff640 1 --2- 172.21.15.111:0/2399303561 >> v2:172.21.15.153:3300/0 conn(0x7f142c144000 0x7f142c0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f142b7fe640 1 --2- 172.21.15.111:0/2399303561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c06b390 0x7f142c0cd740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f142c0726a0 tx=0x7f142002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3e2e522fae0c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f142affd640 1 -- 172.21.15.111:0/2399303561 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420037070 con 0x7f142c06b390 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f142affd640 1 -- 172.21.15.111:0/2399303561 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1420032440 con 0x7f142c06b390 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f142affd640 1 -- 172.21.15.111:0/2399303561 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420032810 con 0x7f142c06b390 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 -- 172.21.15.111:0/2399303561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c06b390 msgr2=0x7f142c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 --2- 172.21.15.111:0/2399303561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c06b390 0x7f142c0cd740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f142c0726a0 tx=0x7f142002a270 comp rx=0 tx=0).stop 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 -- 172.21.15.111:0/2399303561 shutdown_connections 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 --2- 172.21.15.111:0/2399303561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c06b390 0x7f142c0cd740 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 --2- 172.21.15.111:0/2399303561 >> v2:172.21.15.153:3300/0 conn(0x7f142c144000 0x7f142c0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 -- 172.21.15.111:0/2399303561 >> 172.21.15.111:0/2399303561 conn(0x7f142c0b1c00 msgr2=0x7f142c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 -- 172.21.15.111:0/2399303561 shutdown_connections 2024-05-27T00:39:47.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.647+0000 7f1432412640 1 -- 172.21.15.111:0/2399303561 wait complete. 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.648+0000 7f1432412640 1 Processor -- start 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 -- start start 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f142c06b390 0x7f142c0a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 0x7f142c154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f142c144000 0x7f142c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f142c072de0 con 0x7f142c0cdc80 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f142c0b3f80 con 0x7f142c06b390 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f1432412640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f142c072f60 con 0x7f142c144000 2024-05-27T00:39:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.649+0000 7f142b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 0x7f142c154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 0x7f142c154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51400/0 (socket says 172.21.15.111:51400) 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 -- 172.21.15.111:0/1496887026 learned_addr learned my addr 172.21.15.111:0/1496887026 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142bfff640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f142c06b390 0x7f142c0a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f14309af640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f142c144000 0x7f142c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 -- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f142c144000 msgr2=0x7f142c158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f142c144000 0x7f142c158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 -- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f142c06b390 msgr2=0x7f142c0a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f142c06b390 0x7f142c0a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 -- 172.21.15.111:0/1496887026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c1554b0 con 0x7f142c0cdc80 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f14309af640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f142c144000 0x7f142c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f142b7fe640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 0x7f142c154f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1420000c00 tx=0x7f14200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420037040 con 0x7f142c0cdc80 2024-05-27T00:39:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1420004030 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420004400 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c0aa3f0 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.650+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c0aa9b0 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.651+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c068680 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.652+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f142003a070 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.652+0000 7f1428ff9640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f141806be30 0x7f141806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.652+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f14200ab6f0 con 0x7f142c0cdc80 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.652+0000 7f142bfff640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f141806be30 0x7f141806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.653+0000 7f142bfff640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f141806be30 0x7f141806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f142c137e10 tx=0x7f141c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.656+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1420041020 con 0x7f142c0cdc80 2024-05-27T00:39:47.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.828+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f142c159f40 con 0x7f142c0cdc80 2024-05-27T00:39:47.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.828+0000 7f1428ff9640 1 -- 172.21.15.111:0/1496887026 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f1420079710 con 0x7f142c0cdc80 2024-05-27T00:39:47.830 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:47.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f141806be30 msgr2=0x7f141806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f141806be30 0x7f141806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f142c137e10 tx=0x7f141c008040 comp rx=0 tx=0).stop 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 msgr2=0x7f142c154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 0x7f142c154f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1420000c00 tx=0x7f14200046c0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 shutdown_connections 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f141806be30 0x7f141806e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f142c144000 0x7f142c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c0cdc80 0x7f142c154f70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 --2- 172.21.15.111:0/1496887026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f142c06b390 0x7f142c0a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 >> 172.21.15.111:0/1496887026 conn(0x7f142c0b1c00 msgr2=0x7f142c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 shutdown_connections 2024-05-27T00:39:47.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.830+0000 7f1432412640 1 -- 172.21.15.111:0/1496887026 wait complete. 2024-05-27T00:39:47.843 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:47.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.941+0000 7f24f6314640 1 Processor -- start 2024-05-27T00:39:47.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.941+0000 7f24f6314640 1 -- start start 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f6314640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f6314640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24f00726d0 0x7f24f0072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f6314640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f00b3440 con 0x7f24f006b390 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f6314640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f24f0073080 con 0x7f24f00726d0 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f6314640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f24f00b42f0 con 0x7f24f0144000 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24f00726d0 0x7f24f0072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24effff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24f00726d0 0x7f24f0072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56770/0 (socket says 172.21.15.111:56770) 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 -- 172.21.15.111:0/142084398 learned_addr learned my addr 172.21.15.111:0/142084398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24ef7fe640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f24f006b390 0x7f24f00cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47468/0 (socket says 172.21.15.111:47468) 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 -- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 msgr2=0x7f24f00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 --2- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f00b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 -- 172.21.15.111:0/142084398 >> v1:172.21.15.153:6789/0 conn(0x7f24f006b390 legacy=0x7f24f00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24f48b1640 1 -- 172.21.15.111:0/142084398 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f00b36e0 con 0x7f24f00726d0 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.942+0000 7f24eeffd640 1 -- 172.21.15.111:0/142084398 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664987635 0 0) 0x7f24f00b3440 con 0x7f24f006b390 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.943+0000 7f24f48b1640 1 --2- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f00726d0 0x7f24f0072b40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009e50 tx=0x7f24e802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc0d850846ffa2a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.943+0000 7f24eeffd640 1 -- 172.21.15.111:0/142084398 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8036070 con 0x7f24f00726d0 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.943+0000 7f24eeffd640 1 -- 172.21.15.111:0/142084398 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8031600 con 0x7f24f00726d0 2024-05-27T00:39:47.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.943+0000 7f24eeffd640 1 -- 172.21.15.111:0/142084398 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80319d0 con 0x7f24f00726d0 2024-05-27T00:39:47.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 -- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f00726d0 msgr2=0x7f24f0072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 --2- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f00726d0 0x7f24f0072b40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009e50 tx=0x7f24e802a710 comp rx=0 tx=0).stop 2024-05-27T00:39:47.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 -- 172.21.15.111:0/142084398 shutdown_connections 2024-05-27T00:39:47.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 --2- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f00726d0 0x7f24f0072b40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 --2- 172.21.15.111:0/142084398 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f00b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 -- 172.21.15.111:0/142084398 >> 172.21.15.111:0/142084398 conn(0x7f24f00b1c00 msgr2=0x7f24f00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 -- 172.21.15.111:0/142084398 shutdown_connections 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.944+0000 7f24f6314640 1 -- 172.21.15.111:0/142084398 wait complete. 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.945+0000 7f24f6314640 1 Processor -- start 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.945+0000 7f24f6314640 1 -- start start 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f6314640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f006b390 0x7f24f0132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f6314640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 0x7f24f006be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f6314640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f6314640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f24f00b3f80 con 0x7f24f00726d0 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f6314640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f24f00b3ca0 con 0x7f24f006b390 2024-05-27T00:39:47.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f6314640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f24f00cde50 con 0x7f24f0144000 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 0x7f24f006be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 0x7f24f006be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51410/0 (socket says 172.21.15.111:51410) 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 -- 172.21.15.111:0/3332144561 learned_addr learned my addr 172.21.15.111:0/3332144561 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f48b1640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24effff640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f006b390 0x7f24f0132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 -- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 msgr2=0x7f24f01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 -- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f006b390 msgr2=0x7f24f0132b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f006b390 0x7f24f0132b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 -- 172.21.15.111:0/3332144561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f0133080 con 0x7f24f00726d0 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24f48b1640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.946+0000 7f24ef7fe640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 0x7f24f006be90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c410 tx=0x7f24e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.947+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e40074e0 con 0x7f24f00726d0 2024-05-27T00:39:47.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.947+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e40047e0 con 0x7f24f00726d0 2024-05-27T00:39:47.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.947+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4004bd0 con 0x7f24f00726d0 2024-05-27T00:39:47.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.947+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f00aba30 con 0x7f24f00726d0 2024-05-27T00:39:47.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.947+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f00ac270 con 0x7f24f00726d0 2024-05-27T00:39:47.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.948+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0068680 con 0x7f24f00726d0 2024-05-27T00:39:47.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.948+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24e4011070 con 0x7f24f00726d0 2024-05-27T00:39:47.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.949+0000 7f24ecff9640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f24dc06be30 0x7f24dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:47.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.949+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24e408ac90 con 0x7f24f00726d0 2024-05-27T00:39:47.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.949+0000 7f24effff640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f24dc06be30 0x7f24dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:47.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.949+0000 7f24effff640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f24dc06be30 0x7f24dc06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009b40 tx=0x7f24e0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:47.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:47.953+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f24f00aba30 con 0x7f24f00726d0 2024-05-27T00:39:48.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.162+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f0142310 con 0x7f24f00726d0 2024-05-27T00:39:48.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.163+0000 7f24ecff9640 1 -- 172.21.15.111:0/3332144561 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f24f0133080 con 0x7f24f00726d0 2024-05-27T00:39:48.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:48.164 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:39:48.164 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f24dc06be30 msgr2=0x7f24dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f24dc06be30 0x7f24dc06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009b40 tx=0x7f24e0009190 comp rx=0 tx=0).stop 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 msgr2=0x7f24f006be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 0x7f24f006be90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c410 tx=0x7f24e400c8d0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 shutdown_connections 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f24dc06be30 0x7f24dc06e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f24f0144000 0x7f24f01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f00726d0 0x7f24f006be90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 --2- 172.21.15.111:0/3332144561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24f006b390 0x7f24f0132b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 >> 172.21.15.111:0/3332144561 conn(0x7f24f00b1c00 msgr2=0x7f24f0141970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:48.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 shutdown_connections 2024-05-27T00:39:48.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.165+0000 7f24f6314640 1 -- 172.21.15.111:0/3332144561 wait complete. 2024-05-27T00:39:48.177 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:48.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.275+0000 7f54f38fc640 1 Processor -- start 2024-05-27T00:39:48.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.275+0000 7f54f38fc640 1 -- start start 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f38fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f54ec151740 0x7f54ec071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f38fc640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f54ec071b40 0x7f54ec06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f38fc640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54ec06c570 con 0x7f54ec06c160 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f38fc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f54ec071fe0 con 0x7f54ec151740 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f38fc640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f54ec0722c0 con 0x7f54ec071b40 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f54ec151740 0x7f54ec071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f54ec151740 0x7f54ec071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51418/0 (socket says 172.21.15.111:51418) 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 -- 172.21.15.111:0/1852136483 learned_addr learned my addr 172.21.15.111:0/1852136483 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f0e97640 1 --2- 172.21.15.111:0/1852136483 >> v2:172.21.15.153:3300/0 conn(0x7f54ec071b40 0x7f54ec06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 -- 172.21.15.111:0/1852136483 >> v2:172.21.15.153:3300/0 conn(0x7f54ec071b40 msgr2=0x7f54ec06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 --2- 172.21.15.111:0/1852136483 >> v2:172.21.15.153:3300/0 conn(0x7f54ec071b40 0x7f54ec06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 -- 172.21.15.111:0/1852136483 >> v1:172.21.15.153:6789/0 conn(0x7f54ec06c160 legacy=0x7f54ec0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f1698640 1 -- 172.21.15.111:0/1852136483 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec0725a0 con 0x7f54ec151740 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.276+0000 7f54f0e97640 1 --2- 172.21.15.111:0/1852136483 >> v2:172.21.15.153:3300/0 conn(0x7f54ec071b40 0x7f54ec06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.277+0000 7f54f1698640 1 --2- 172.21.15.111:0/1852136483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec151740 0x7f54ec071600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0ccd00 tx=0x7f54e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6829db0f0733b26d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.277+0000 7f54e3fff640 1 -- 172.21.15.111:0/1852136483 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8036070 con 0x7f54ec151740 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.277+0000 7f54e3fff640 1 -- 172.21.15.111:0/1852136483 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e8031600 con 0x7f54ec151740 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.277+0000 7f54e3fff640 1 -- 172.21.15.111:0/1852136483 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80319d0 con 0x7f54ec151740 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.277+0000 7f54f38fc640 1 -- 172.21.15.111:0/1852136483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec151740 msgr2=0x7f54ec071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.277+0000 7f54f38fc640 1 --2- 172.21.15.111:0/1852136483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec151740 0x7f54ec071600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0ccd00 tx=0x7f54e802a6f0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.278+0000 7f54f38fc640 1 -- 172.21.15.111:0/1852136483 shutdown_connections 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.278+0000 7f54f38fc640 1 --2- 172.21.15.111:0/1852136483 >> v2:172.21.15.153:3300/0 conn(0x7f54ec071b40 0x7f54ec06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.278+0000 7f54f38fc640 1 --2- 172.21.15.111:0/1852136483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec151740 0x7f54ec071600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.278+0000 7f54f38fc640 1 -- 172.21.15.111:0/1852136483 >> 172.21.15.111:0/1852136483 conn(0x7f54ec0b1c00 msgr2=0x7f54ec0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:48.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.278+0000 7f54f38fc640 1 -- 172.21.15.111:0/1852136483 shutdown_connections 2024-05-27T00:39:48.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.278+0000 7f54f38fc640 1 -- 172.21.15.111:0/1852136483 wait complete. 2024-05-27T00:39:48.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.279+0000 7f54f38fc640 1 Processor -- start 2024-05-27T00:39:48.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.279+0000 7f54f38fc640 1 -- start start 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec06c160 0x7f54ec0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 0x7f54ec152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec151740 0x7f54ec150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec15af10 con 0x7f54ec071b40 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec151b20 con 0x7f54ec151740 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec15b090 con 0x7f54ec06c160 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 0x7f54ec152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 0x7f54ec152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51432/0 (socket says 172.21.15.111:51432) 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 -- 172.21.15.111:0/3119775659 learned_addr learned my addr 172.21.15.111:0/3119775659 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f1e99640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec151740 0x7f54ec150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f1698640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec06c160 0x7f54ec0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 -- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec06c160 msgr2=0x7f54ec0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec06c160 0x7f54ec0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 -- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec151740 msgr2=0x7f54ec150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec151740 0x7f54ec150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 -- 172.21.15.111:0/3119775659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec0ad810 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f0e97640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 0x7f54ec152660 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00c7f0 tx=0x7f54dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc015070 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc007a30 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc007e00 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.280+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec0adaf0 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.281+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec0ae390 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.281+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ec068680 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.282+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54dc018070 con 0x7f54ec071b40 2024-05-27T00:39:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.282+0000 7f54e1ffb640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54d006be30 0x7f54d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.282+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f54dc08b0e0 con 0x7f54ec071b40 2024-05-27T00:39:48.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.282+0000 7f54f1698640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54d006be30 0x7f54d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.283+0000 7f54f1698640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54d006be30 0x7f54d006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54e8033000 tx=0x7f54e8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.286+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f54dc059100 con 0x7f54ec071b40 2024-05-27T00:39:48.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.462+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54ec152ba0 con 0x7f54ec071b40 2024-05-27T00:39:48.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.463+0000 7f54e1ffb640 1 -- 172.21.15.111:0/3119775659 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f54dc05d400 con 0x7f54ec071b40 2024-05-27T00:39:48.464 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:48.464 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:48.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54d006be30 msgr2=0x7f54d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54d006be30 0x7f54d006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54e8033000 tx=0x7f54e8009000 comp rx=0 tx=0).stop 2024-05-27T00:39:48.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 msgr2=0x7f54ec152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 0x7f54ec152660 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00c7f0 tx=0x7f54dc00cbf0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 shutdown_connections 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54d006be30 0x7f54d006e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec151740 0x7f54ec150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54ec071b40 0x7f54ec152660 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 --2- 172.21.15.111:0/3119775659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54ec06c160 0x7f54ec0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 >> 172.21.15.111:0/3119775659 conn(0x7f54ec0b1c00 msgr2=0x7f54ec0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 shutdown_connections 2024-05-27T00:39:48.467 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.465+0000 7f54f38fc640 1 -- 172.21.15.111:0/3119775659 wait complete. 2024-05-27T00:39:48.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:39:48.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:48.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.579+0000 7f1d3f28c640 1 Processor -- start 2024-05-27T00:39:48.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.579+0000 7f1d3f28c640 1 -- start start 2024-05-27T00:39:48.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3f28c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d380d19f0 0x7f1d380ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3f28c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d380c7700 con 0x7f1d380ceeb0 2024-05-27T00:39:48.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3f28c640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d380c79e0 con 0x7f1d381459c0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3f28c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1d380c7cc0 con 0x7f1d380d19f0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3d829640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1d380ceeb0 0x7f1d380cc950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36786/0 (socket says 172.21.15.111:36786) 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3d829640 1 -- 172.21.15.111:0/542360053 learned_addr learned my addr 172.21.15.111:0/542360053 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3c827640 1 --2- 172.21.15.111:0/542360053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d380d19f0 0x7f1d380ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371504599 0 0) 0x7f1d380c7700 con 0x7f1d380ceeb0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d24003520 con 0x7f1d380ceeb0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900206475 0 0) 0x7f1d380c79e0 con 0x7f1d381459c0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d380c7700 con 0x7f1d381459c0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3687623246 0 0) 0x7f1d24003520 con 0x7f1d380ceeb0 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3c827640 1 -- 172.21.15.111:0/542360053 >> v1:172.21.15.153:6789/0 conn(0x7f1d381459c0 legacy=0x7f1d380c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3c827640 1 -- 172.21.15.111:0/542360053 >> v1:172.21.15.111:6789/0 conn(0x7f1d380ceeb0 legacy=0x7f1d380cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.580+0000 7f1d3c827640 1 -- 172.21.15.111:0/542360053 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d380c7fa0 con 0x7f1d380d19f0 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.581+0000 7f1d3c827640 1 --2- 172.21.15.111:0/542360053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d380d19f0 0x7f1d380ce970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d280097d0 tx=0x7f1d2802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ad7d650d533fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.581+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28037070 con 0x7f1d380d19f0 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.581+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2802ab90 con 0x7f1d380d19f0 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.581+0000 7f1d2ffff640 1 -- 172.21.15.111:0/542360053 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340019f0 con 0x7f1d380d19f0 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 -- 172.21.15.111:0/542360053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d380d19f0 msgr2=0x7f1d380ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/542360053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d380d19f0 0x7f1d380ce970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d280097d0 tx=0x7f1d2802a100 comp rx=0 tx=0).stop 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 -- 172.21.15.111:0/542360053 shutdown_connections 2024-05-27T00:39:48.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/542360053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d380d19f0 0x7f1d380ce970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 -- 172.21.15.111:0/542360053 >> 172.21.15.111:0/542360053 conn(0x7f1d380d1f60 msgr2=0x7f1d380d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:48.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 -- 172.21.15.111:0/542360053 shutdown_connections 2024-05-27T00:39:48.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.582+0000 7f1d3f28c640 1 -- 172.21.15.111:0/542360053 wait complete. 2024-05-27T00:39:48.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.583+0000 7f1d3f28c640 1 Processor -- start 2024-05-27T00:39:48.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.583+0000 7f1d3f28c640 1 -- start start 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3f28c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 0x7f1d38133d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3f28c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d380d19f0 0x7f1d3806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3f28c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d381459c0 0x7f1d381e4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3f28c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d380cd1e0 con 0x7f1d380ceeb0 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3f28c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1d380cd060 con 0x7f1d380d19f0 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3f28c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1d380cd360 con 0x7f1d381459c0 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 0x7f1d38133d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 0x7f1d38133d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51438/0 (socket says 172.21.15.111:51438) 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 -- 172.21.15.111:0/3897153875 learned_addr learned my addr 172.21.15.111:0/3897153875 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d829640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d381459c0 0x7f1d381e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3c827640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d380d19f0 0x7f1d3806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 -- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d381459c0 msgr2=0x7f1d381e4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d381459c0 0x7f1d381e4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 -- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d380d19f0 msgr2=0x7f1d3806f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d380d19f0 0x7f1d3806f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 -- 172.21.15.111:0/3897153875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d381342a0 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3c827640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d380d19f0 0x7f1d3806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d3d028640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 0x7f1d38133d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c430 tx=0x7f1d3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30017070 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.584+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d300052f0 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.585+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30003830 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.585+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d38131640 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.585+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d38131ee0 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.586+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d38068680 con 0x7f1d380ceeb0 2024-05-27T00:39:48.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.586+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d3001a070 con 0x7f1d380ceeb0 2024-05-27T00:39:48.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.587+0000 7f1d2dffb640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.587+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1d3008a620 con 0x7f1d380ceeb0 2024-05-27T00:39:48.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.587+0000 7f1d3c827640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.587+0000 7f1d3c827640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d28034000 tx=0x7f1d2802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.591+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1d381342a0 con 0x7f1d380ceeb0 2024-05-27T00:39:48.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.765+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d380d2340 con 0x7f1d380ceeb0 2024-05-27T00:39:48.766 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.765+0000 7f1d2dffb640 1 -- 172.21.15.111:0/3897153875 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f1d30058c90 con 0x7f1d380ceeb0 2024-05-27T00:39:48.767 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:48.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d1c06be30 msgr2=0x7f1d1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d28034000 tx=0x7f1d2802e000 comp rx=0 tx=0).stop 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 msgr2=0x7f1d38133d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 0x7f1d38133d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c430 tx=0x7f1d3000c8f0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 shutdown_connections 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1d381459c0 0x7f1d381e4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d380d19f0 0x7f1d3806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 --2- 172.21.15.111:0/3897153875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d380ceeb0 0x7f1d38133d60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 >> 172.21.15.111:0/3897153875 conn(0x7f1d380d1f60 msgr2=0x7f1d380c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 shutdown_connections 2024-05-27T00:39:48.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.767+0000 7f1d3f28c640 1 -- 172.21.15.111:0/3897153875 wait complete. 2024-05-27T00:39:48.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:39:48.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.880+0000 7f4aa16df640 1 Processor -- start 2024-05-27T00:39:48.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.880+0000 7f4aa16df640 1 -- start start 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4aa16df640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4a9c144000 0x7f4a9c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4aa16df640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a9c06b390 0x7f4a9c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4aa16df640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c0cdc80 0x7f4a9c0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4aa16df640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9c0b42f0 con 0x7f4a9c144000 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4aa16df640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9c0b3440 con 0x7f4a9c06b390 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4aa16df640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b36e0 con 0x7f4a9c0cdc80 2024-05-27T00:39:48.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4a9c144000 0x7f4a9c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4a9c144000 0x7f4a9c0b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51446/0 (socket says 172.21.15.111:51446) 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 -- 172.21.15.111:0/1398591470 learned_addr learned my addr 172.21.15.111:0/1398591470 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9b7fe640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c0cdc80 0x7f4a9c0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9a7fc640 1 --2- 172.21.15.111:0/1398591470 >> v2:172.21.15.153:3300/0 conn(0x7f4a9c06b390 0x7f4a9c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 -- 172.21.15.111:0/1398591470 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c0cdc80 msgr2=0x7f4a9c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c0cdc80 0x7f4a9c0726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 -- 172.21.15.111:0/1398591470 >> v2:172.21.15.153:3300/0 conn(0x7f4a9c06b390 msgr2=0x7f4a9c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:48.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 --2- 172.21.15.111:0/1398591470 >> v2:172.21.15.153:3300/0 conn(0x7f4a9c06b390 0x7f4a9c0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9affd640 1 -- 172.21.15.111:0/1398591470 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c0b39c0 con 0x7f4a9c144000 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9a7fc640 1 --2- 172.21.15.111:0/1398591470 >> v2:172.21.15.153:3300/0 conn(0x7f4a9c06b390 0x7f4a9c0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.881+0000 7f4a9b7fe640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c0cdc80 0x7f4a9c0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.882+0000 7f4a9affd640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c144000 0x7f4a9c0b2f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c072650 tx=0x7f4a8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8be0b4286d1bba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.882+0000 7f4a99ffb640 1 -- 172.21.15.111:0/1398591470 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c036070 con 0x7f4a9c144000 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.882+0000 7f4a99ffb640 1 -- 172.21.15.111:0/1398591470 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c031600 con 0x7f4a9c144000 2024-05-27T00:39:48.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.882+0000 7f4a99ffb640 1 -- 172.21.15.111:0/1398591470 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0319d0 con 0x7f4a9c144000 2024-05-27T00:39:48.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 -- 172.21.15.111:0/1398591470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c144000 msgr2=0x7f4a9c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c144000 0x7f4a9c0b2f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c072650 tx=0x7f4a8c02a6a0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 -- 172.21.15.111:0/1398591470 shutdown_connections 2024-05-27T00:39:48.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c0cdc80 0x7f4a9c0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 --2- 172.21.15.111:0/1398591470 >> v2:172.21.15.153:3300/0 conn(0x7f4a9c06b390 0x7f4a9c0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 --2- 172.21.15.111:0/1398591470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c144000 0x7f4a9c0b2f00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 -- 172.21.15.111:0/1398591470 >> 172.21.15.111:0/1398591470 conn(0x7f4a9c0b1c00 msgr2=0x7f4a9c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:48.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 -- 172.21.15.111:0/1398591470 shutdown_connections 2024-05-27T00:39:48.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.883+0000 7f4aa16df640 1 -- 172.21.15.111:0/1398591470 wait complete. 2024-05-27T00:39:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.884+0000 7f4aa16df640 1 Processor -- start 2024-05-27T00:39:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.884+0000 7f4aa16df640 1 -- start start 2024-05-27T00:39:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4aa16df640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 0x7f4a9c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4aa16df640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c0cdc80 0x7f4a9c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4aa16df640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c144000 0x7f4a9c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4aa16df640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c072c10 con 0x7f4a9c06b390 2024-05-27T00:39:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4aa16df640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b3f80 con 0x7f4a9c0cdc80 2024-05-27T00:39:48.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4aa16df640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c072d90 con 0x7f4a9c144000 2024-05-27T00:39:48.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 0x7f4a9c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 0x7f4a9c14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51460/0 (socket says 172.21.15.111:51460) 2024-05-27T00:39:48.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 -- 172.21.15.111:0/3859810890 learned_addr learned my addr 172.21.15.111:0/3859810890 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:48.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9a7fc640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c0cdc80 0x7f4a9c14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9b7fe640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c144000 0x7f4a9c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 -- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c144000 msgr2=0x7f4a9c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c144000 0x7f4a9c0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 -- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c0cdc80 msgr2=0x7f4a9c14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c0cdc80 0x7f4a9c14dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 -- 172.21.15.111:0/3859810890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c14f100 con 0x7f4a9c06b390 2024-05-27T00:39:48.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9a7fc640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c0cdc80 0x7f4a9c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:48.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.885+0000 7f4a9affd640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 0x7f4a9c14d6a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c034040 tx=0x7f4a8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.886+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c036050 con 0x7f4a9c06b390 2024-05-27T00:39:48.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.886+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c042070 con 0x7f4a9c06b390 2024-05-27T00:39:48.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.886+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c14f3e0 con 0x7f4a9c06b390 2024-05-27T00:39:48.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.886+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009510 con 0x7f4a9c06b390 2024-05-27T00:39:48.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.886+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c14f9a0 con 0x7f4a9c06b390 2024-05-27T00:39:48.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.887+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c068680 con 0x7f4a9c06b390 2024-05-27T00:39:48.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.887+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a8c007c30 con 0x7f4a9c06b390 2024-05-27T00:39:48.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.887+0000 7f4a837fe640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a8806be30 0x7f4a8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:48.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.888+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4a8c0ac100 con 0x7f4a9c06b390 2024-05-27T00:39:48.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.888+0000 7f4a9a7fc640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a8806be30 0x7f4a8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:48.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.888+0000 7f4a9a7fc640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a8806be30 0x7f4a8806e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0cc040 tx=0x7f4a90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:48.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:48.892+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4a8c045360 con 0x7f4a9c06b390 2024-05-27T00:39:49.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.069+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a9c0af830 con 0x7f4a9c06b390 2024-05-27T00:39:49.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.069+0000 7f4a837fe640 1 -- 172.21.15.111:0/3859810890 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f4a8c07a800 con 0x7f4a9c06b390 2024-05-27T00:39:49.070 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:49.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a8806be30 msgr2=0x7f4a8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a8806be30 0x7f4a8806e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0cc040 tx=0x7f4a90008040 comp rx=0 tx=0).stop 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 msgr2=0x7f4a9c14d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 0x7f4a9c14d6a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c034040 tx=0x7f4a8c004a40 comp rx=0 tx=0).stop 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 shutdown_connections 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a8806be30 0x7f4a8806e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a9c144000 0x7f4a9c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a9c0cdc80 0x7f4a9c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 --2- 172.21.15.111:0/3859810890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a9c06b390 0x7f4a9c14d6a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 >> 172.21.15.111:0/3859810890 conn(0x7f4a9c0b1c00 msgr2=0x7f4a9c0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 shutdown_connections 2024-05-27T00:39:49.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.071+0000 7f4aa16df640 1 -- 172.21.15.111:0/3859810890 wait complete. 2024-05-27T00:39:49.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:39:49.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.185+0000 7f6842dc1640 1 Processor -- start 2024-05-27T00:39:49.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.185+0000 7f6842dc1640 1 -- start start 2024-05-27T00:39:49.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f6842dc1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f683c0b3630 0x7f683c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f6842dc1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f683c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f6842dc1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f683c06cc70 con 0x7f683c0b77e0 2024-05-27T00:39:49.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f6842dc1640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f683c0d1e50 con 0x7f683c0b3630 2024-05-27T00:39:49.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f6842dc1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f683c0d2130 con 0x7f683c162560 2024-05-27T00:39:49.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f683c0b77e0 0x7f683c161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36790/0 (socket says 172.21.15.111:36790) 2024-05-27T00:39:49.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683bfff640 1 -- 172.21.15.111:0/1390169875 learned_addr learned my addr 172.21.15.111:0/1390169875 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:49.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f6840b5d640 1 --2- 172.21.15.111:0/1390169875 >> v2:172.21.15.153:3300/0 conn(0x7f683c0b3630 0x7f683c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131833913 0 0) 0x7f683c06cc70 con 0x7f683c0b77e0 2024-05-27T00:39:49.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f682c003520 con 0x7f683c0b77e0 2024-05-27T00:39:49.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f684135e640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f683c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 109146567 0 0) 0x7f682c003520 con 0x7f683c0b77e0 2024-05-27T00:39:49.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 msgr2=0x7f683c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f683c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 >> v2:172.21.15.153:3300/0 conn(0x7f683c0b3630 msgr2=0x7f683c0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 --2- 172.21.15.111:0/1390169875 >> v2:172.21.15.153:3300/0 conn(0x7f683c0b3630 0x7f683c0b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c0d2410 con 0x7f683c0b77e0 2024-05-27T00:39:49.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f6828001670 con 0x7f683c0b77e0 2024-05-27T00:39:49.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 >> v1:172.21.15.111:6789/0 conn(0x7f683c0b77e0 legacy=0x7f683c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f682c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f682c004800 0x7f682c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f682c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f683c0d1e50 con 0x7f682c0039b0 2024-05-27T00:39:49.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.186+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f683c0d2130 con 0x7f682c004800 2024-05-27T00:39:49.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f682c00b9c0 con 0x7f683c162560 2024-05-27T00:39:49.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f682c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f683bfff640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f682c004800 0x7f682c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f684135e640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f682c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 -- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 msgr2=0x7f682c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f682c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 -- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f682c004800 msgr2=0x7f682c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f682c004800 0x7f682c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 -- 172.21.15.111:0/1390169875 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c003520 con 0x7f682c0039b0 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f684135e640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c162560 0x7f682c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.187+0000 7f6840b5d640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f682c003dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6830009ac0 tx=0x7f682c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6e5746c292042c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830036070 con 0x7f682c0039b0 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6830031400 con 0x7f682c0039b0 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f683b7fe640 1 -- 172.21.15.111:0/1390169875 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830009690 con 0x7f682c0039b0 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f6842dc1640 1 -- 172.21.15.111:0/1390169875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 msgr2=0x7f682c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f6842dc1640 1 --2- 172.21.15.111:0/1390169875 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f682c003dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6830009ac0 tx=0x7f682c005830 comp rx=0 tx=0).stop 2024-05-27T00:39:49.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f684135e640 1 -- 172.21.15.111:0/1390169875 reap_dead start 2024-05-27T00:39:49.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f6842dc1640 1 -- 172.21.15.111:0/1390169875 shutdown_connections 2024-05-27T00:39:49.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f6842dc1640 1 -- 172.21.15.111:0/1390169875 >> 172.21.15.111:0/1390169875 conn(0x7f683c0d42c0 msgr2=0x7f683c0d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:49.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f6842dc1640 1 -- 172.21.15.111:0/1390169875 shutdown_connections 2024-05-27T00:39:49.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.188+0000 7f6842dc1640 1 -- 172.21.15.111:0/1390169875 wait complete. 2024-05-27T00:39:49.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.189+0000 7f6842dc1640 1 Processor -- start 2024-05-27T00:39:49.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.189+0000 7f6842dc1640 1 -- start start 2024-05-27T00:39:49.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6842dc1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f683c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6842dc1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c0b8a00 0x7f683c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6842dc1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f683c0cd8c0 0x7f683c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6842dc1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f683c0d2d90 con 0x7f682c0039b0 2024-05-27T00:39:49.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6842dc1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f683c0d2c10 con 0x7f683c0cd8c0 2024-05-27T00:39:49.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6842dc1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f683c0d2f10 con 0x7f683c0b8a00 2024-05-27T00:39:49.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6840b5d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f683c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6840b5d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f683c1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51478/0 (socket says 172.21.15.111:51478) 2024-05-27T00:39:49.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.190+0000 7f6840b5d640 1 -- 172.21.15.111:0/4232957063 learned_addr learned my addr 172.21.15.111:0/4232957063 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:49.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f684135e640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f683c0cd8c0 0x7f683c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f683bfff640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c0b8a00 0x7f683c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6840b5d640 1 -- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c0b8a00 msgr2=0x7f683c0cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6840b5d640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c0b8a00 0x7f683c0cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6840b5d640 1 -- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f683c0cd8c0 msgr2=0x7f683c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6840b5d640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f683c0cd8c0 0x7f683c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6840b5d640 1 -- 172.21.15.111:0/4232957063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c0c9930 con 0x7f682c0039b0 2024-05-27T00:39:49.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f683bfff640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c0b8a00 0x7f683c0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6840b5d640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f683c1571f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6830034040 tx=0x7f6830004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830040020 con 0x7f682c0039b0 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6830003ac0 con 0x7f682c0039b0 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c0c9c10 con 0x7f682c0039b0 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830036040 con 0x7f682c0039b0 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.191+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c0ca1d0 con 0x7f682c0039b0 2024-05-27T00:39:49.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.192+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c0ce4b0 con 0x7f682c0039b0 2024-05-27T00:39:49.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.192+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6830043020 con 0x7f682c0039b0 2024-05-27T00:39:49.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.193+0000 7f68397fa640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f682006be60 0x7f682006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.193+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68300ab1e0 con 0x7f682c0039b0 2024-05-27T00:39:49.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.193+0000 7f683bfff640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f682006be60 0x7f682006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.194+0000 7f683bfff640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f682006be60 0x7f682006e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6828000fb0 tx=0x7f6828009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.197+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f683c0ce4b0 con 0x7f682c0039b0 2024-05-27T00:39:49.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.373+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f683c0cdcd0 con 0x7f682c0039b0 2024-05-27T00:39:49.374 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.373+0000 7f68397fa640 1 -- 172.21.15.111:0/4232957063 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f683c0cdcd0 con 0x7f682c0039b0 2024-05-27T00:39:49.374 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:39:49.376 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f682006be60 msgr2=0x7f682006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f682006be60 0x7f682006e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6828000fb0 tx=0x7f6828009040 comp rx=0 tx=0).stop 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 msgr2=0x7f683c1571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f683c1571f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6830034040 tx=0x7f6830004730 comp rx=0 tx=0).stop 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 shutdown_connections 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f682006be60 0x7f682006e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f683c0cd8c0 0x7f683c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f683c0b8a00 0x7f683c0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 --2- 172.21.15.111:0/4232957063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682c0039b0 0x7f683c1571f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 >> 172.21.15.111:0/4232957063 conn(0x7f683c0d42c0 msgr2=0x7f683c0cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 shutdown_connections 2024-05-27T00:39:49.377 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.375+0000 7f6842dc1640 1 -- 172.21.15.111:0/4232957063 wait complete. 2024-05-27T00:39:49.387 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:39:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.488+0000 7fdd3c33b640 1 Processor -- start 2024-05-27T00:39:49.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.488+0000 7fdd3c33b640 1 -- start start 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.489+0000 7fdd3c33b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 0x7fdd3406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.489+0000 7fdd3c33b640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdd340cf030 0x7fdd340cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.489+0000 7fdd3c33b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd34071d10 con 0x7fdd34151740 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.489+0000 7fdd3c33b640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdd34071fe0 con 0x7fdd340cf030 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd3c33b640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd340722c0 con 0x7fdd3406bc20 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd3a0d7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdd34151740 0x7fdd34071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36794/0 (socket says 172.21.15.111:36794) 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd3a0d7640 1 -- 172.21.15.111:0/968100219 learned_addr learned my addr 172.21.15.111:0/968100219 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:49.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd398d6640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 0x7fdd3406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444423323 0 0) 0x7fdd34071d10 con 0x7fdd34151740 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd3a8d8640 1 --2- 172.21.15.111:0/968100219 >> v2:172.21.15.153:3300/0 conn(0x7fdd340cf030 0x7fdd340cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd20003520 con 0x7fdd34151740 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3887511931 0 0) 0x7fdd20003520 con 0x7fdd34151740 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 msgr2=0x7fdd3406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 0x7fdd3406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 >> v2:172.21.15.153:3300/0 conn(0x7fdd340cf030 msgr2=0x7fdd340cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 --2- 172.21.15.111:0/968100219 >> v2:172.21.15.153:3300/0 conn(0x7fdd340cf030 0x7fdd340cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd340725a0 con 0x7fdd34151740 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fdd30001670 con 0x7fdd34151740 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 >> v1:172.21.15.111:6789/0 conn(0x7fdd34151740 legacy=0x7fdd34071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd200039b0 0x7fdd20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd20004800 0x7fdd20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 0x7fdd200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34071fe0 con 0x7fdd200039b0 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340722c0 con 0x7fdd20004800 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2000cad0 con 0x7fdd3406bc20 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd200039b0 0x7fdd20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd3a8d8640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 0x7fdd200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.490+0000 7fdd398d6640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd20004800 0x7fdd20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd3a0d7640 1 -- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 msgr2=0x7fdd200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd3406bc20 0x7fdd200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd3a0d7640 1 -- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd20004800 msgr2=0x7fdd20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd20004800 0x7fdd20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd3a0d7640 1 -- 172.21.15.111:0/968100219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd20003520 con 0x7fdd200039b0 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd200039b0 0x7fdd20003dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007fd0 tx=0x7fdd20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e19537ceac1dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30007d50 con 0x7fdd200039b0 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd30005ad0 con 0x7fdd200039b0 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.491+0000 7fdd390d5640 1 -- 172.21.15.111:0/968100219 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30007070 con 0x7fdd200039b0 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3c33b640 1 -- 172.21.15.111:0/968100219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd200039b0 msgr2=0x7fdd20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/968100219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd200039b0 0x7fdd20003dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007fd0 tx=0x7fdd20005830 comp rx=0 tx=0).stop 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3a8d8640 1 -- 172.21.15.111:0/968100219 reap_dead start 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3c33b640 1 -- 172.21.15.111:0/968100219 shutdown_connections 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3c33b640 1 -- 172.21.15.111:0/968100219 >> 172.21.15.111:0/968100219 conn(0x7fdd340b1c00 msgr2=0x7fdd340b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3c33b640 1 -- 172.21.15.111:0/968100219 shutdown_connections 2024-05-27T00:39:49.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.492+0000 7fdd3c33b640 1 -- 172.21.15.111:0/968100219 wait complete. 2024-05-27T00:39:49.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.493+0000 7fdd3c33b640 1 Processor -- start 2024-05-27T00:39:49.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.493+0000 7fdd3c33b640 1 -- start start 2024-05-27T00:39:49.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3c33b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd200039b0 0x7fdd34150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3c33b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd341ef680 0x7fdd34157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3c33b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 0x7fdd340b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3c33b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3415af10 con 0x7fdd34145230 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3c33b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34151b20 con 0x7fdd341ef680 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3c33b640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3415b090 con 0x7fdd200039b0 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3a8d8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 0x7fdd340b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3a8d8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 0x7fdd340b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51490/0 (socket says 172.21.15.111:51490) 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3a8d8640 1 -- 172.21.15.111:0/3399291996 learned_addr learned my addr 172.21.15.111:0/3399291996 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd398d6640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd341ef680 0x7fdd34157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.494+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd200039b0 0x7fdd34150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a8d8640 1 -- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd200039b0 msgr2=0x7fdd34150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a8d8640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd200039b0 0x7fdd34150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a8d8640 1 -- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd341ef680 msgr2=0x7fdd34157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a8d8640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd341ef680 0x7fdd34157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a8d8640 1 -- 172.21.15.111:0/3399291996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd34152e00 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd398d6640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd341ef680 0x7fdd34157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd200039b0 0x7fdd34150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3a8d8640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 0x7fdd340b4050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd340734e0 tx=0x7fdd2c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00fde0 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c015430 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341530e0 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c015820 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.495+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341536a0 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.496+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd34068680 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.496+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd2c0159c0 con 0x7fdd34145230 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.497+0000 7fdd2affd640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdd1806bdd0 0x7fdd1806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.497+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd2c08bff0 con 0x7fdd34145230 2024-05-27T00:39:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.497+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdd1806bdd0 0x7fdd1806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.498+0000 7fdd3a0d7640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdd1806bdd0 0x7fdd1806e290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd30033000 tx=0x7fdd30009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.501+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdd2c019030 con 0x7fdd34145230 2024-05-27T00:39:49.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.677+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdd341582a0 con 0x7fdd34145230 2024-05-27T00:39:49.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.677+0000 7fdd2affd640 1 -- 172.21.15.111:0/3399291996 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fdd2c010040 con 0x7fdd34145230 2024-05-27T00:39:49.678 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:49.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdd1806bdd0 msgr2=0x7fdd1806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdd1806bdd0 0x7fdd1806e290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd30033000 tx=0x7fdd30009030 comp rx=0 tx=0).stop 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 msgr2=0x7fdd340b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 0x7fdd340b4050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd340734e0 tx=0x7fdd2c00f580 comp rx=0 tx=0).stop 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 shutdown_connections 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdd1806bdd0 0x7fdd1806e290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd34145230 0x7fdd340b4050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd341ef680 0x7fdd34157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 --2- 172.21.15.111:0/3399291996 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd200039b0 0x7fdd34150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 >> 172.21.15.111:0/3399291996 conn(0x7fdd340b1c00 msgr2=0x7fdd3415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 shutdown_connections 2024-05-27T00:39:49.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.679+0000 7fdd3c33b640 1 -- 172.21.15.111:0/3399291996 wait complete. 2024-05-27T00:39:49.692 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:49.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.790+0000 7fe932858640 1 Processor -- start 2024-05-27T00:39:49.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.790+0000 7fe932858640 1 -- start start 2024-05-27T00:39:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe932858640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe932858640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe92c13f340 0x7fe92c0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe932858640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c142650 con 0x7fe92c0cce00 2024-05-27T00:39:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe932858640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe92c0cf210 con 0x7fe92c13f340 2024-05-27T00:39:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe932858640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c0cebc0 con 0x7fe92c13ef60 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe923fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe92c13f340 0x7fe92c0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe923fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe92c13f340 0x7fe92c0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51500/0 (socket says 172.21.15.111:51500) 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe930df5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe92c0cce00 0x7fe92c141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36808/0 (socket says 172.21.15.111:36808) 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.791+0000 7fe923fff640 1 -- 172.21.15.111:0/1236668085 learned_addr learned my addr 172.21.15.111:0/1236668085 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe923fff640 1 -- 172.21.15.111:0/1236668085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 msgr2=0x7fe92c0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe923fff640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c0cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe923fff640 1 -- 172.21.15.111:0/1236668085 >> v1:172.21.15.111:6789/0 conn(0x7fe92c0cce00 legacy=0x7fe92c141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe923fff640 1 -- 172.21.15.111:0/1236668085 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0ceea0 con 0x7fe92c13f340 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe92bfff640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c0cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe923fff640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c13f340 0x7fe92c0cc8c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe914009870 tx=0x7fe91402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c538e823228796 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe92b7fe640 1 -- 172.21.15.111:0/1236668085 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914037070 con 0x7fe92c13f340 2024-05-27T00:39:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe92b7fe640 1 -- 172.21.15.111:0/1236668085 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe914032440 con 0x7fe92c13f340 2024-05-27T00:39:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.792+0000 7fe92b7fe640 1 -- 172.21.15.111:0/1236668085 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914032810 con 0x7fe92c13f340 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 -- 172.21.15.111:0/1236668085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c13f340 msgr2=0x7fe92c0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c13f340 0x7fe92c0cc8c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe914009870 tx=0x7fe91402a270 comp rx=0 tx=0).stop 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 -- 172.21.15.111:0/1236668085 shutdown_connections 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c13f340 0x7fe92c0cc8c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 --2- 172.21.15.111:0/1236668085 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c0cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 -- 172.21.15.111:0/1236668085 >> 172.21.15.111:0/1236668085 conn(0x7fe92c0b1c00 msgr2=0x7fe92c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 -- 172.21.15.111:0/1236668085 shutdown_connections 2024-05-27T00:39:49.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.793+0000 7fe932858640 1 -- 172.21.15.111:0/1236668085 wait complete. 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.794+0000 7fe932858640 1 Processor -- start 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.794+0000 7fe932858640 1 -- start start 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe932858640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 0x7fe92c12ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe932858640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe932858640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c13f340 0x7fe92c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe932858640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c142030 con 0x7fe92c0cce00 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe932858640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c0ce890 con 0x7fe92c13f340 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe932858640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c1421b0 con 0x7fe92c13ef60 2024-05-27T00:39:49.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 0x7fe92c12ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 0x7fe92c12ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51502/0 (socket says 172.21.15.111:51502) 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 -- 172.21.15.111:0/2654859361 learned_addr learned my addr 172.21.15.111:0/2654859361 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 -- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 msgr2=0x7fe92c155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe923fff640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe930df5640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c13f340 0x7fe92c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 -- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c13f340 msgr2=0x7fe92c15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c13f340 0x7fe92c15e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.795+0000 7fe92bfff640 1 -- 172.21.15.111:0/2654859361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c15a410 con 0x7fe92c0cce00 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.796+0000 7fe92bfff640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 0x7fe92c12ef60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c820 tx=0x7fe91c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.796+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c015070 con 0x7fe92c0cce00 2024-05-27T00:39:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.796+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c007a30 con 0x7fe92c0cce00 2024-05-27T00:39:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.796+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c007e00 con 0x7fe92c0cce00 2024-05-27T00:39:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.796+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c15a6f0 con 0x7fe92c0cce00 2024-05-27T00:39:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.796+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c15acb0 con 0x7fe92c0cce00 2024-05-27T00:39:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.797+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c068680 con 0x7fe92c0cce00 2024-05-27T00:39:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.797+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe91c018070 con 0x7fe92c0cce00 2024-05-27T00:39:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.798+0000 7fe9297fa640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe90c06be60 0x7fe90c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.798+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe91c08b0b0 con 0x7fe92c0cce00 2024-05-27T00:39:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.798+0000 7fe923fff640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe90c06be60 0x7fe90c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.798+0000 7fe923fff640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe90c06be60 0x7fe90c06e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe914034000 tx=0x7fe91402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:49.802+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe91c0597b0 con 0x7fe92c0cce00 2024-05-27T00:39:50.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.013+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe92c145ba0 con 0x7fe92c0cce00 2024-05-27T00:39:50.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.014+0000 7fe9297fa640 1 -- 172.21.15.111:0/2654859361 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7fe91c0590b0 con 0x7fe92c0cce00 2024-05-27T00:39:50.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:50.015 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:39:50.015 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:39:50.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe90c06be60 msgr2=0x7fe90c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe90c06be60 0x7fe90c06e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe914034000 tx=0x7fe91402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:50.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 msgr2=0x7fe92c12ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 0x7fe92c12ef60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c820 tx=0x7fe91c00cc20 comp rx=0 tx=0).stop 2024-05-27T00:39:50.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 shutdown_connections 2024-05-27T00:39:50.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe90c06be60 0x7fe90c06e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe92c13f340 0x7fe92c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe92c13ef60 0x7fe92c155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 --2- 172.21.15.111:0/2654859361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c0cce00 0x7fe92c12ef60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 >> 172.21.15.111:0/2654859361 conn(0x7fe92c0b1c00 msgr2=0x7fe92c0cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:50.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 shutdown_connections 2024-05-27T00:39:50.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.016+0000 7fe932858640 1 -- 172.21.15.111:0/2654859361 wait complete. 2024-05-27T00:39:50.028 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.127+0000 7f4134f7e640 1 Processor -- start 2024-05-27T00:39:50.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.127+0000 7f4134f7e640 1 -- start start 2024-05-27T00:39:50.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f4134f7e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4130144000 0x7f41300b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f4134f7e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41300b3440 con 0x7f41300726d0 2024-05-27T00:39:50.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f4134f7e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41300b36e0 con 0x7f413006b390 2024-05-27T00:39:50.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f4134f7e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f41300b42f0 con 0x7f4130144000 2024-05-27T00:39:50.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412ed9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f41300726d0 0x7f4130072ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36824/0 (socket says 172.21.15.111:36824) 2024-05-27T00:39:50.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412ed9d640 1 -- 172.21.15.111:0/1089395820 learned_addr learned my addr 172.21.15.111:0/1089395820 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412e59c640 1 --2- 172.21.15.111:0/1089395820 >> v2:172.21.15.153:3300/0 conn(0x7f4130144000 0x7f41300b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851332063 0 0) 0x7f41300b3440 con 0x7f41300726d0 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4118003520 con 0x7f41300726d0 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2273475725 0 0) 0x7f4118003520 con 0x7f41300726d0 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 >> v2:172.21.15.153:3300/0 conn(0x7f4130144000 msgr2=0x7f41300b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412d59a640 1 --2- 172.21.15.111:0/1089395820 >> v2:172.21.15.153:3300/0 conn(0x7f4130144000 0x7f41300b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.128+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 >> v1:172.21.15.153:6789/0 conn(0x7f413006b390 legacy=0x7f41300cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41300b39c0 con 0x7f41300726d0 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f41280019f0 con 0x7f41300726d0 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 >> v1:172.21.15.111:6789/0 conn(0x7f41300726d0 legacy=0x7f4130072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41180039b0 0x7f4118003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f4118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41180051c0 0x7f411800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41300b42f0 con 0x7f4118004800 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f411800c090 con 0x7f41180051c0 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f411800c290 con 0x7f41180039b0 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412dd9b640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f4118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412ed9d640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41180051c0 0x7f411800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412e59c640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41180039b0 0x7f4118003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412dd9b640 1 -- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41180039b0 msgr2=0x7f4118003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412dd9b640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41180039b0 0x7f4118003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412dd9b640 1 -- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41180051c0 msgr2=0x7f411800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:50.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412dd9b640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41180051c0 0x7f411800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412dd9b640 1 -- 172.21.15.111:0/1089395820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4118003520 con 0x7f4118004800 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412e59c640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f41180039b0 0x7f4118003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412ed9d640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41180051c0 0x7f411800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.129+0000 7f412ed9d640 1 -- 172.21.15.111:0/1089395820 reap_dead start 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f412dd9b640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f4118004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4124001720 tx=0x7f4118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460b0b84e617703e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124001370 con 0x7f4118004800 2024-05-27T00:39:50.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41240052d0 con 0x7f4118004800 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f412d59a640 1 -- 172.21.15.111:0/1089395820 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412402ab90 con 0x7f4118004800 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 -- 172.21.15.111:0/1089395820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 msgr2=0x7f4118004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f4118004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4124001720 tx=0x7f4118005830 comp rx=0 tx=0).stop 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 -- 172.21.15.111:0/1089395820 shutdown_connections 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 --2- 172.21.15.111:0/1089395820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f4118004c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 -- 172.21.15.111:0/1089395820 >> 172.21.15.111:0/1089395820 conn(0x7f41300b1c00 msgr2=0x7f41300b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 -- 172.21.15.111:0/1089395820 shutdown_connections 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.130+0000 7f4134f7e640 1 -- 172.21.15.111:0/1089395820 wait complete. 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 Processor -- start 2024-05-27T00:39:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 -- start start 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f413015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f413015ac00 0x7f41301f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413006bfe0 0x7f413006c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41300b2f00 con 0x7f4118004800 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f41300b3f80 con 0x7f413006bfe0 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f4134f7e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f41300b3080 con 0x7f413015ac00 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.132+0000 7f412e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f413015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f413015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51528/0 (socket says 172.21.15.111:51528) 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 -- 172.21.15.111:0/2796467072 learned_addr learned my addr 172.21.15.111:0/2796467072 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412dd9b640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f413015ac00 0x7f41301f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412ed9d640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413006bfe0 0x7f413006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 -- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f413015ac00 msgr2=0x7f41301f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f413015ac00 0x7f41301f3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 -- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413006bfe0 msgr2=0x7f413006c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413006bfe0 0x7f413006c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 -- 172.21.15.111:0/2796467072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413015aa10 con 0x7f4118004800 2024-05-27T00:39:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412ed9d640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413006bfe0 0x7f413006c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f412e59c640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f413015a4d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4120010a30 tx=0x7f4120010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000f040 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412000ec80 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412001c400 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41301f3a70 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.133+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41301f3ff0 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.134+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413006c9a0 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.135+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f412001c5a0 con 0x7f4118004800 2024-05-27T00:39:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.135+0000 7f411effd640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f411006be30 0x7f411006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.135+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f412008b810 con 0x7f4118004800 2024-05-27T00:39:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.135+0000 7f412dd9b640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f411006be30 0x7f411006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.136+0000 7f412dd9b640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f411006be30 0x7f411006e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4124001720 tx=0x7f412402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:50.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.140+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4120018030 con 0x7f4118004800 2024-05-27T00:39:50.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.315+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4130068680 con 0x7f4118004800 2024-05-27T00:39:50.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.316+0000 7f411effd640 1 -- 172.21.15.111:0/2796467072 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f4120089e20 con 0x7f4118004800 2024-05-27T00:39:50.317 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:50.318 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f411006be30 msgr2=0x7f411006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f411006be30 0x7f411006e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4124001720 tx=0x7f412402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 msgr2=0x7f413015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f413015a4d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4120010a30 tx=0x7f4120010ef0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 shutdown_connections 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f411006be30 0x7f411006e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f413006bfe0 0x7f413006c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f413015ac00 0x7f41301f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 --2- 172.21.15.111:0/2796467072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4118004800 0x7f413015a4d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 >> 172.21.15.111:0/2796467072 conn(0x7f41300b1c00 msgr2=0x7f4130071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:50.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 shutdown_connections 2024-05-27T00:39:50.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.318+0000 7f4134f7e640 1 -- 172.21.15.111:0/2796467072 wait complete. 2024-05-27T00:39:50.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:39:50.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.429+0000 7fd368212640 1 Processor -- start 2024-05-27T00:39:50.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.429+0000 7fd368212640 1 -- start start 2024-05-27T00:39:50.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd368212640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd36013ef60 0x7fd3600cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd368212640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd3600cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd368212640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd360142650 con 0x7fd3600cce00 2024-05-27T00:39:50.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd368212640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd3600cf210 con 0x7fd36013ef60 2024-05-27T00:39:50.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd368212640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600cebc0 con 0x7fd36013f340 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd3657ad640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd3600cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd365fae640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd36013ef60 0x7fd3600cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd3657ad640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd3600cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:48372/0 (socket says 172.21.15.111:48372) 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd3667af640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd3600cce00 0x7fd360141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:47510/0 (socket says 172.21.15.111:47510) 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd365fae640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd36013ef60 0x7fd3600cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:56924/0 (socket says 172.21.15.111:56924) 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.430+0000 7fd3657ad640 1 -- 172.21.15.111:0/70428364 learned_addr learned my addr 172.21.15.111:0/70428364 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.431+0000 7fd365fae640 1 -- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 msgr2=0x7fd3600cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.431+0000 7fd365fae640 1 --2- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd3600cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.431+0000 7fd365fae640 1 -- 172.21.15.111:0/70428364 >> v1:172.21.15.153:6789/0 conn(0x7fd3600cce00 legacy=0x7fd360141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.431+0000 7fd364fac640 1 -- 172.21.15.111:0/70428364 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3774409962 0 0) 0x7fd360142650 con 0x7fd3600cce00 2024-05-27T00:39:50.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.431+0000 7fd365fae640 1 -- 172.21.15.111:0/70428364 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3600ceea0 con 0x7fd36013ef60 2024-05-27T00:39:50.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.431+0000 7fd3657ad640 1 --2- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd3600cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd365fae640 1 --2- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36013ef60 0x7fd3600cf6a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3540097a0 tx=0x7fd35402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81cdb425c8ce0b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd364fac640 1 -- 172.21.15.111:0/70428364 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354037070 con 0x7fd36013ef60 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd364fac640 1 -- 172.21.15.111:0/70428364 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd354032440 con 0x7fd36013ef60 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd364fac640 1 -- 172.21.15.111:0/70428364 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354032810 con 0x7fd36013ef60 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 -- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36013ef60 msgr2=0x7fd3600cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 --2- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36013ef60 0x7fd3600cf6a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3540097a0 tx=0x7fd35402a180 comp rx=0 tx=0).stop 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 -- 172.21.15.111:0/70428364 shutdown_connections 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 --2- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd3600cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 --2- 172.21.15.111:0/70428364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36013ef60 0x7fd3600cf6a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 -- 172.21.15.111:0/70428364 >> 172.21.15.111:0/70428364 conn(0x7fd3600b1c00 msgr2=0x7fd3600b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:50.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 -- 172.21.15.111:0/70428364 shutdown_connections 2024-05-27T00:39:50.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.432+0000 7fd368212640 1 -- 172.21.15.111:0/70428364 wait complete. 2024-05-27T00:39:50.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.433+0000 7fd368212640 1 Processor -- start 2024-05-27T00:39:50.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.433+0000 7fd368212640 1 -- start start 2024-05-27T00:39:50.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd368212640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3600cce00 0x7fd3600721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd368212640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 0x7fd3600726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd368212640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd36013f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd368212640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd360142030 con 0x7fd36013ef60 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd368212640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600ce890 con 0x7fd3600cce00 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd368212640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd3601421b0 con 0x7fd36013f340 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd3657ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 0x7fd3600726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd3657ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 0x7fd3600726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51536/0 (socket says 172.21.15.111:51536) 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd3657ad640 1 -- 172.21.15.111:0/200606272 learned_addr learned my addr 172.21.15.111:0/200606272 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.434+0000 7fd3667af640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd36013f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd365fae640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3600cce00 0x7fd3600721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3657ad640 1 -- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 msgr2=0x7fd36013f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3657ad640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd36013f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3657ad640 1 -- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3600cce00 msgr2=0x7fd3600721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3657ad640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3600cce00 0x7fd3600721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3657ad640 1 -- 172.21.15.111:0/200606272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3600addb0 con 0x7fd36013ef60 2024-05-27T00:39:50.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3657ad640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 0x7fd3600726f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00efc0 tx=0x7fd35c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:50.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00ce60 con 0x7fd36013ef60 2024-05-27T00:39:50.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c019070 con 0x7fd36013ef60 2024-05-27T00:39:50.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3600ae090 con 0x7fd36013ef60 2024-05-27T00:39:50.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c014680 con 0x7fd36013ef60 2024-05-27T00:39:50.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.435+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3600ae650 con 0x7fd36013ef60 2024-05-27T00:39:50.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.436+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36006fb70 con 0x7fd36013ef60 2024-05-27T00:39:50.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.436+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd35c005d10 con 0x7fd36013ef60 2024-05-27T00:39:50.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.437+0000 7fd3527fc640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:50.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.437+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd35c08b5f0 con 0x7fd36013ef60 2024-05-27T00:39:50.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.437+0000 7fd365fae640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:50.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.437+0000 7fd365fae640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd34406be30 0x7fd34406e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd35402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:50.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.441+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd36006fb70 con 0x7fd36013ef60 2024-05-27T00:39:50.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.650+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3600d0da0 con 0x7fd36013ef60 2024-05-27T00:39:50.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.650+0000 7fd3527fc640 1 -- 172.21.15.111:0/200606272 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7fd3600d0da0 con 0x7fd36013ef60 2024-05-27T00:39:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:50.652 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:39:50.653 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:39:50.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.652+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd34406be30 msgr2=0x7fd34406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.652+0000 7fd368212640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd34406be30 0x7fd34406e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd35402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:50.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.652+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 msgr2=0x7fd3600726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:50.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.652+0000 7fd368212640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 0x7fd3600726f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00efc0 tx=0x7fd35c00c490 comp rx=0 tx=0).stop 2024-05-27T00:39:50.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 shutdown_connections 2024-05-27T00:39:50.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd34406be30 0x7fd34406e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd36013f340 0x7fd36013f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd36013ef60 0x7fd3600726f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 --2- 172.21.15.111:0/200606272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd3600cce00 0x7fd3600721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 >> 172.21.15.111:0/200606272 conn(0x7fd3600b1c00 msgr2=0x7fd3600b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 shutdown_connections 2024-05-27T00:39:50.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:50.653+0000 7fd368212640 1 -- 172.21.15.111:0/200606272 wait complete. 2024-05-27T00:39:51.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T00:39:20.908117+0000', 'modified': '2024-05-27T00:39:37.972042+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4320}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4320': {'gid': 4320, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6836/2203621473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 2203621473}, {'type': 'v1', 'addr': '172.21.15.153:6836', 'nonce': 2203621473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4320, 'qdb_cluster': [4320]} 2024-05-27T00:39:51.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T00:39:51.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-27T00:39:51.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.172+0000 7f63d05ce640 1 Processor -- start 2024-05-27T00:39:51.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.172+0000 7f63d05ce640 1 -- start start 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.173+0000 7f63d05ce640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 0x7f63c81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.173+0000 7f63d05ce640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63c8160b30 con 0x7f63c806b390 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.173+0000 7f63d05ce640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63c80cec10 con 0x7f63c815c250 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.173+0000 7f63d05ce640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f63c8160db0 con 0x7f63c80cdc10 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.173+0000 7f63ceb6b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f63c806b390 0x7f63c80cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59034/0 (socket says 172.21.15.111:59034) 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.173+0000 7f63ceb6b640 1 -- 172.21.15.111:0/4201377383 learned_addr learned my addr 172.21.15.111:0/4201377383 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349002607 0 0) 0x7f63c0001110 con 0x7f63c806b390 2024-05-27T00:39:51.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63b0003520 con 0x7f63c806b390 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cdb69640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 0x7f63c81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2538618612 0 0) 0x7f63b0003520 con 0x7f63c806b390 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 msgr2=0x7f63c81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 0x7f63c81605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 >> v1:172.21.15.153:6789/0 conn(0x7f63c815c250 legacy=0x7f63c815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c80b3e50 con 0x7f63c806b390 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f63c0001840 con 0x7f63c806b390 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 >> v1:172.21.15.111:6789/0 conn(0x7f63c806b390 legacy=0x7f63c80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 0x7f63b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b0004d40 0x7f63b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8160db0 con 0x7f63b00039e0 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63b000baf0 con 0x7f63b0004d40 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f63b000bcf0 con 0x7f63c80cdc10 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ce36a640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63cdb69640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 0x7f63b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b0004d40 0x7f63b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ce36a640 1 -- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 msgr2=0x7f63b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ce36a640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c80cdc10 0x7f63b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ce36a640 1 -- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b0004d40 msgr2=0x7f63b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ce36a640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b0004d40 0x7f63b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ce36a640 1 -- 172.21.15.111:0/4201377383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b0003520 con 0x7f63b00039e0 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.174+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63b0004d40 0x7f63b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.175+0000 7f63ce36a640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63b0003e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f63b8001e50 tx=0x7f63b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be058916c2c335c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.175+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8001a70 con 0x7f63b00039e0 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.175+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b802ada0 con 0x7f63b00039e0 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.175+0000 7f63cd368640 1 -- 172.21.15.111:0/4201377383 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003bb0 con 0x7f63b00039e0 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63d05ce640 1 -- 172.21.15.111:0/4201377383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 msgr2=0x7f63b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63d05ce640 1 --2- 172.21.15.111:0/4201377383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63b0003e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f63b8001e50 tx=0x7f63b0005830 comp rx=0 tx=0).stop 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63ceb6b640 1 -- 172.21.15.111:0/4201377383 reap_dead start 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63d05ce640 1 -- 172.21.15.111:0/4201377383 shutdown_connections 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63d05ce640 1 -- 172.21.15.111:0/4201377383 >> 172.21.15.111:0/4201377383 conn(0x7f63c8071b20 msgr2=0x7f63c80b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63d05ce640 1 -- 172.21.15.111:0/4201377383 shutdown_connections 2024-05-27T00:39:51.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.176+0000 7f63d05ce640 1 -- 172.21.15.111:0/4201377383 wait complete. 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.177+0000 7f63d05ce640 1 Processor -- start 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.177+0000 7f63d05ce640 1 -- start start 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63d05ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63c8150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63d05ce640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c8099960 0x7f63c8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63d05ce640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c8157510 0x7f63c814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63d05ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63c815c630 con 0x7f63b00039e0 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63d05ce640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63c80b4130 con 0x7f63c8157510 2024-05-27T00:39:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63d05ce640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f63c815c7b0 con 0x7f63c8099960 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63c8150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63c8150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45848/0 (socket says 172.21.15.111:45848) 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 -- 172.21.15.111:0/2013284294 learned_addr learned my addr 172.21.15.111:0/2013284294 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 -- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c8099960 msgr2=0x7f63c8156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63cdb69640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c8099960 0x7f63c8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c8157510 0x7f63c814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c8099960 0x7f63c8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 -- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c8157510 msgr2=0x7f63c814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c8157510 0x7f63c814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 -- 172.21.15.111:0/2013284294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c8099de0 con 0x7f63b00039e0 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c8157510 0x7f63c814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63ce36a640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63c8150e80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f63b8001e80 tx=0x7f63b80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63c6ffd640 1 -- 172.21.15.111:0/2013284294 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8007db0 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.178+0000 7f63c6ffd640 1 -- 172.21.15.111:0/2013284294 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b8040070 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.179+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c814fde0 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.179+0000 7f63c6ffd640 1 -- 172.21.15.111:0/2013284294 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8001a70 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.179+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81503a0 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.181+0000 7f63c6ffd640 1 -- 172.21.15.111:0/2013284294 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63b803a070 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.181+0000 7f63c6ffd640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63b406be30 0x7f63b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.181+0000 7f63c6ffd640 1 -- 172.21.15.111:0/2013284294 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f63b80aae10 con 0x7f63b00039e0 2024-05-27T00:39:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.181+0000 7f63cdb69640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63b406be30 0x7f63b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.182+0000 7f63cdb69640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63b406be30 0x7f63b406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63c81420c0 tx=0x7f63bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.182+0000 7f63d33bb740 1 Processor -- start 2024-05-27T00:39:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.182+0000 7f63d33bb740 1 -- start start 2024-05-27T00:39:51.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f546d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f546da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f543d960 0x5555f5441170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63d33bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555f53a6740 con 0x5555f546b0e0 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5555f5344a80 con 0x5555f543d960 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x5555f53a2700 con 0x5555f546ad00 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f546da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f546da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45856/0 (socket says 172.21.15.111:45856) 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 -- 172.21.15.111:0/3265491703 learned_addr learned my addr 172.21.15.111:0/3265491703 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f543d960 0x5555f5441170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63ce36a640 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f546d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 -- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 msgr2=0x5555f546d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f546d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 -- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f543d960 msgr2=0x5555f5441170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f543d960 0x5555f5441170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 -- 172.21.15.111:0/3265491703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f536ffb0 con 0x5555f546b0e0 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f543d960 0x5555f5441170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63cdb69640 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f546da50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f63bc013c80 tx=0x7f63bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e257fa2a72917c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63affff640 1 -- 172.21.15.111:0/3265491703 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc013cb0 con 0x5555f546b0e0 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.183+0000 7f63affff640 1 -- 172.21.15.111:0/3265491703 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63bc028070 con 0x5555f546b0e0 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63affff640 1 -- 172.21.15.111:0/3265491703 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc023640 con 0x5555f546b0e0 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 -- 172.21.15.111:0/3265491703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 msgr2=0x5555f546da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f546da50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f63bc013c80 tx=0x7f63bc016000 comp rx=0 tx=0).stop 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 -- 172.21.15.111:0/3265491703 shutdown_connections 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f543d960 0x5555f5441170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f546da50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 --2- 172.21.15.111:0/3265491703 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f546d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 -- 172.21.15.111:0/3265491703 >> 172.21.15.111:0/3265491703 conn(0x5555f543fbd0 msgr2=0x5555f5488510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 -- 172.21.15.111:0/3265491703 shutdown_connections 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.184+0000 7f63d33bb740 1 -- 172.21.15.111:0/3265491703 wait complete. 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.185+0000 7f63d33bb740 1 -- start start 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.185+0000 7f63d33bb740 1 Processor -- start 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f548b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f5468560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f548e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555f5442d80 con 0x5555f546b0e0 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5555f532fc30 con 0x5555f548cbe0 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x5555f5443060 con 0x5555f546ad00 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63d33bb740 10 client.0 _ll_register_callbacks cb 0x5555f53617f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T00:39:51.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f5468560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f5468560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45862/0 (socket says 172.21.15.111:45862) 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 -- 172.21.15.111:0/78260581 learned_addr learned my addr 172.21.15.111:0/78260581 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63ce36a640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f548b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f548e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 -- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 msgr2=0x5555f548b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f548b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 -- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 msgr2=0x5555f548e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f548e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 -- 172.21.15.111:0/78260581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f50ba2d0 con 0x5555f546b0e0 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f548e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63cdb69640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f5468560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f63bc024000 tx=0x7f63bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7e9d96053e0360 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.186+0000 7f63ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-05-27T00:39:51.196 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63ad7fa640 1 -- 172.21.15.111:0/78260581 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc03a060 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63ad7fa640 1 -- 172.21.15.111:0/78260581 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f63bc00f040 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63bc028070 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63ad7fa640 1 -- 172.21.15.111:0/78260581 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f63bc040050 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 10 client.4905 fetch_fsmap learned FSMap version 9 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5555f51846b0 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63ad7fa640 1 -- 172.21.15.111:0/78260581 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63bc028070 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63ad7fa640 1 -- 172.21.15.111:0/78260581 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f63bc043020 con 0x5555f546b0e0 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 10 client.4905 fetch_fsmap finished waiting for FSMap version 9 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 10 client.4905 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 20 client.4905 populate_metadata read hostname 'smithi111' 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f54a4b50 0x5555f54a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 4 client.4905 mds_command: new command op to 4320 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 --> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f53b3e70 con 0x5555f54a4b50 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.187+0000 7f63ce36a640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f54a4b50 0x5555f54a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.188+0000 7f63ce36a640 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f54a4b50 0x5555f54a6f40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e319fde7f3df32 server_cookie=f34eb9599d9e2c14 in_seq=0 out_seq=0 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.188+0000 7f63ad7fa640 10 client.4905 ms_handle_connect on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.188+0000 7f63ad7fa640 1 -- 172.21.15.111:0/78260581 <== mds.0 v2:172.21.15.153:6834/2203621473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555f53b3e70 con 0x5555f54a4b50 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.188+0000 7f63ad7fa640 10 client.4905 handle_command_reply: tid=0 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.188+0000 7f63d33bb740 1 client.4905 shutdown 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 msgr2=0x5555f5468560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f5468560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f63bc024000 tx=0x7f63bc004730 comp rx=0 tx=0).stop 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 shutdown_connections 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f54a4b50 0x5555f54a6f40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f548e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546b0e0 0x5555f5468560 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63ad7fa640 0 client.4905 ms_handle_reset on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:51.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 --2- 172.21.15.111:0/78260581 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546ad00 0x5555f548b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 >> 172.21.15.111:0/78260581 conn(0x5555f543d960 msgr2=0x5555f546d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 shutdown_connections 2024-05-27T00:39:51.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 1 -- 172.21.15.111:0/78260581 wait complete. 2024-05-27T00:39:51.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.189+0000 7f63d33bb740 20 client.4905 trim_cache size 0 max 16384 2024-05-27T00:39:51.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.207+0000 7f63d33bb740 1 Processor -- start 2024-05-27T00:39:51.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.207+0000 7f63d33bb740 1 -- start start 2024-05-27T00:39:51.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 0x5555f5485ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5486420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f5442180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63d33bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81fd470 con 0x5555f546ad00 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81fc7d0 con 0x5555f548cbe0 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f63c81fce20 con 0x5555f546b0e0 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 0x5555f5485ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 0x5555f5485ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45870/0 (socket says 172.21.15.111:45870) 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 -- 172.21.15.111:0/450904848 learned_addr learned my addr 172.21.15.111:0/450904848 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 -- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 msgr2=0x5555f5486420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5486420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f5442180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 -- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 msgr2=0x5555f5442180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f5442180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 -- 172.21.15.111:0/450904848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f5468480 con 0x5555f546ad00 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.208+0000 7f63ce36a640 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 0x5555f5485ee0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63b8078d10 tx=0x7f63b8075150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8506c03ee855e18f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63af7fe640 1 -- 172.21.15.111:0/450904848 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8077080 con 0x5555f546ad00 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63af7fe640 1 -- 172.21.15.111:0/450904848 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b80a7070 con 0x5555f546ad00 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63af7fe640 1 -- 172.21.15.111:0/450904848 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80753f0 con 0x5555f546ad00 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 -- 172.21.15.111:0/450904848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 msgr2=0x5555f5485ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 0x5555f5485ee0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63b8078d10 tx=0x7f63b8075150 comp rx=0 tx=0).stop 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 -- 172.21.15.111:0/450904848 shutdown_connections 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f548cbe0 0x5555f5442180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5486420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 --2- 172.21.15.111:0/450904848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f546ad00 0x5555f5485ee0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 -- 172.21.15.111:0/450904848 >> 172.21.15.111:0/450904848 conn(0x5555f543d960 msgr2=0x5555f5443990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 -- 172.21.15.111:0/450904848 shutdown_connections 2024-05-27T00:39:51.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.209+0000 7f63d33bb740 1 -- 172.21.15.111:0/450904848 wait complete. 2024-05-27T00:39:51.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.210+0000 7f63d33bb740 1 -- start start 2024-05-27T00:39:51.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.210+0000 7f63d33bb740 1 Processor -- start 2024-05-27T00:39:51.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f546ad00 0x5555f54653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5465920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 0x5555f548f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555f548f180 con 0x5555f548cbe0 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x5555f548ac30 con 0x5555f546ad00 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x5555f5470dd0 con 0x5555f546b0e0 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 10 client.0 _ll_register_callbacks cb 0x7f63bc0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 0x5555f548f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 0x5555f548f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45882/0 (socket says 172.21.15.111:45882) 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 -- 172.21.15.111:0/2950359016 learned_addr learned my addr 172.21.15.111:0/2950359016 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63cdb69640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5465920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ce36a640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f546ad00 0x5555f54653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 -- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 msgr2=0x5555f5465920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5465920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 -- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f546ad00 msgr2=0x5555f54653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f546ad00 0x5555f54653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 -- 172.21.15.111:0/2950359016 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f5439370 con 0x5555f548cbe0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ceb6b640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 0x5555f548f300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63c0001eb0 tx=0x7f63c0011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f831179cdd52f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ad7fa640 1 -- 172.21.15.111:0/2950359016 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0013540 con 0x5555f548cbe0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ad7fa640 1 -- 172.21.15.111:0/2950359016 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f63c00136e0 con 0x5555f548cbe0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b80a7070 con 0x5555f548cbe0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63ad7fa640 1 -- 172.21.15.111:0/2950359016 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f63c0008020 con 0x5555f548cbe0 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.211+0000 7f63d33bb740 10 client.4911 fetch_fsmap learned FSMap version 9 2024-05-27T00:39:51.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5555f5465e60 con 0x5555f548cbe0 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63ad7fa640 1 -- 172.21.15.111:0/2950359016 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b80a7070 con 0x5555f548cbe0 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63ad7fa640 1 -- 172.21.15.111:0/2950359016 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f63c0022080 con 0x5555f548cbe0 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 10 client.4911 fetch_fsmap finished waiting for FSMap version 9 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 10 client.4911 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 20 client.4911 populate_metadata read hostname 'smithi111' 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f5487010 0x5555f54873f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 4 client.4911 mds_command: new command op to 4320 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 --> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] -- command(tid 0: {"prefix": "status"}) -- 0x5555f53ae1c0 con 0x5555f5487010 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63ce36a640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f5487010 0x5555f54873f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63ad7fa640 10 client.4911 ms_handle_connect on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.212+0000 7f63ce36a640 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f5487010 0x5555f54873f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df974366893aa69d server_cookie=fd0823a34e9acd72 in_seq=0 out_seq=0 2024-05-27T00:39:51.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63ad7fa640 1 -- 172.21.15.111:0/2950359016 <== mds.0 v2:172.21.15.153:6834/2203621473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5555f53ae1c0 con 0x5555f5487010 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63ad7fa640 10 client.4911 handle_command_reply: tid=0 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 client.4911 shutdown 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 msgr2=0x5555f548f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 0x5555f548f300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63c0001eb0 tx=0x7f63c0011910 comp rx=0 tx=0).stop 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 shutdown_connections 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:6834/2203621473,v1:172.21.15.153:6836/2203621473] conn(0x5555f5487010 0x5555f54873f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555f548cbe0 0x5555f548f300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5555f546b0e0 0x5555f5465920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63ad7fa640 0 client.4911 ms_handle_reset on v2:172.21.15.153:6834/2203621473 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 --2- 172.21.15.111:0/2950359016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5555f546ad00 0x5555f54653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 >> 172.21.15.111:0/2950359016 conn(0x5555f543d960 msgr2=0x5555f548c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 shutdown_connections 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 1 -- 172.21.15.111:0/2950359016 wait complete. 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d33bb740 20 client.4911 trim_cache size 0 max 16384 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "ef97d1dc-a8f7-4487-922a-ccc176f6927a", 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4320, 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-05-27T00:39:51.224 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-05-27T00:39:51.225 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 29.634609542, 2024-05-27T00:39:51.225 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 9, 2024-05-27T00:39:51.225 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 25, 2024-05-27T00:39:51.225 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 25, 2024-05-27T00:39:51.225 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 36.732053618999998 2024-05-27T00:39:51.225 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-27T00:39:51.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63b406be30 msgr2=0x7f63b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d05ce640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63b406be30 0x7f63b406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63c81420c0 tx=0x7f63bc008040 comp rx=0 tx=0).stop 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 msgr2=0x7f63c8150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.213+0000 7f63d05ce640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63c8150e80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f63b8001e80 tx=0x7f63b80014a0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 shutdown_connections 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63b406be30 0x7f63b406e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63c8157510 0x7f63c814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63c8099960 0x7f63c8156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 --2- 172.21.15.111:0/2013284294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63b00039e0 0x7f63c8150e80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 >> 172.21.15.111:0/2013284294 conn(0x7f63c8071b20 msgr2=0x7f63c806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 shutdown_connections 2024-05-27T00:39:51.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.214+0000 7f63d05ce640 1 -- 172.21.15.111:0/2013284294 wait complete. 2024-05-27T00:39:51.228 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T00:39:51.250 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T00:39:51.250 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T00:39:51.250 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T00:39:51.250 INFO:teuthology.run_tasks:Running task kclient... 2024-05-27T00:39:51.269 INFO:tasks.kclient:Mounting kernel clients... 2024-05-27T00:39:51.269 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-27T00:39:51.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.270 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-27T00:39:51.286 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-27T00:39:51.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.287 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T00:39:51.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.353 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-05-27T00:39:51.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.442 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-27T00:39:51.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-05-27T00:39:51.525 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2024-05-27T00:39:51.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:39:51.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.527 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2024-05-27T00:39:51.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.546 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2024-05-27T00:39:51.618 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2024-05-27T00:39:51.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:39:51.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.620 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2024-05-27T00:39:51.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:51.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2024-05-27T00:39:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2024-05-27T00:39:51.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T00:39:51.748 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-27T00:39:51.748 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T00:39:51.749 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T00:39:51.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T00:39:51.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.847+0000 7fb43116a640 1 Processor -- start 2024-05-27T00:39:51.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.847+0000 7fb43116a640 1 -- start start 2024-05-27T00:39:51.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb43116a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c151740 0x7fb42c071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb43116a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb42c071b40 0x7fb42c06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb43116a640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42c071fe0 con 0x7fb42c06c160 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb43116a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb42c06c5d0 con 0x7fb42c071b40 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb43116a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c0722c0 con 0x7fb42c151740 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb42a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb42c071b40 0x7fb42c06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb42a59c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb42c071b40 0x7fb42c06bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45884/0 (socket says 172.21.15.111:45884) 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.848+0000 7fb42a59c640 1 -- 172.21.15.111:0/1437831521 learned_addr learned my addr 172.21.15.111:0/1437831521 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42ad9d640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c151740 0x7fb42c071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42a59c640 1 -- 172.21.15.111:0/1437831521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c151740 msgr2=0x7fb42c071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42a59c640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c151740 0x7fb42c071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42a59c640 1 -- 172.21.15.111:0/1437831521 >> v1:172.21.15.153:6789/0 conn(0x7fb42c06c160 legacy=0x7fb42c0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42a59c640 1 -- 172.21.15.111:0/1437831521 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c0725a0 con 0x7fb42c071b40 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42ad9d640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c151740 0x7fb42c071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:51.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.849+0000 7fb42a59c640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c071b40 0x7fb42c06bc20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb41400bb10 tx=0x7fb41402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650ca8fc30237cbd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb429d9b640 1 -- 172.21.15.111:0/1437831521 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414036020 con 0x7fb42c071b40 2024-05-27T00:39:51.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb429d9b640 1 -- 172.21.15.111:0/1437831521 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414032820 con 0x7fb42c071b40 2024-05-27T00:39:51.852 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb429d9b640 1 -- 172.21.15.111:0/1437831521 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414032bf0 con 0x7fb42c071b40 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 -- 172.21.15.111:0/1437831521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c071b40 msgr2=0x7fb42c06bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c071b40 0x7fb42c06bc20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb41400bb10 tx=0x7fb41402c920 comp rx=0 tx=0).stop 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 -- 172.21.15.111:0/1437831521 shutdown_connections 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c071b40 0x7fb42c06bc20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 --2- 172.21.15.111:0/1437831521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c151740 0x7fb42c071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 -- 172.21.15.111:0/1437831521 >> 172.21.15.111:0/1437831521 conn(0x7fb42c0b1c00 msgr2=0x7fb42c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 -- 172.21.15.111:0/1437831521 shutdown_connections 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.850+0000 7fb43116a640 1 -- 172.21.15.111:0/1437831521 wait complete. 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 Processor -- start 2024-05-27T00:39:51.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 -- start start 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 0x7fb42c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c071b40 0x7fb42c14fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb42c151740 0x7fb42c14c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c15af10 con 0x7fb42c06c160 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c151b20 con 0x7fb42c151740 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb43116a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c15b090 con 0x7fb42c071b40 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.852+0000 7fb42ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 0x7fb42c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 0x7fb42c0ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45898/0 (socket says 172.21.15.111:45898) 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 -- 172.21.15.111:0/903153708 learned_addr learned my addr 172.21.15.111:0/903153708 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:51.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42b59e640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb42c151740 0x7fb42c14c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42a59c640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c071b40 0x7fb42c14fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 -- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c071b40 msgr2=0x7fb42c14fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c071b40 0x7fb42c14fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 -- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb42c151740 msgr2=0x7fb42c14c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb42c151740 0x7fb42c14c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 -- 172.21.15.111:0/903153708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c0ce900 con 0x7fb42c06c160 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb42ad9d640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 0x7fb42c0ce320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c7f0 tx=0x7fb41c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c015070 con 0x7fb42c06c160 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41c007820 con 0x7fb42c06c160 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c007bd0 con 0x7fb42c06c160 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c0cebe0 con 0x7fb42c06c160 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.853+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c0cf1a0 con 0x7fb42c06c160 2024-05-27T00:39:51.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.854+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42c15cae0 con 0x7fb42c06c160 2024-05-27T00:39:51.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.855+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb41c018070 con 0x7fb42c06c160 2024-05-27T00:39:51.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.855+0000 7fb4237fe640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40c06be30 0x7fb40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:51.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.855+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb41c08b210 con 0x7fb42c06c160 2024-05-27T00:39:51.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.855+0000 7fb42a59c640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40c06be30 0x7fb40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:51.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.856+0000 7fb42a59c640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40c06be30 0x7fb40c06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb414037000 tx=0x7fb41403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:51.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:51.859+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb42c15cae0 con 0x7fb42c06c160 2024-05-27T00:39:52.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.071+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb42c0cf570 con 0x7fb42c06c160 2024-05-27T00:39:52.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.071+0000 7fb4237fe640 1 -- 172.21.15.111:0/903153708 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fb41c059910 con 0x7fb42c06c160 2024-05-27T00:39:52.072 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40c06be30 msgr2=0x7fb40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40c06be30 0x7fb40c06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb414037000 tx=0x7fb41403d040 comp rx=0 tx=0).stop 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 msgr2=0x7fb42c0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 0x7fb42c0ce320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c7f0 tx=0x7fb41c00ccb0 comp rx=0 tx=0).stop 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 shutdown_connections 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb40c06be30 0x7fb40c06e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:52.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb42c151740 0x7fb42c14c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:52.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb42c071b40 0x7fb42c14fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:52.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 --2- 172.21.15.111:0/903153708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb42c06c160 0x7fb42c0ce320 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:52.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 >> 172.21.15.111:0/903153708 conn(0x7fb42c0b1c00 msgr2=0x7fb42c15cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:52.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 shutdown_connections 2024-05-27T00:39:52.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52.073+0000 7fb43116a640 1 -- 172.21.15.111:0/903153708 wait complete. 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T00:39:52.085 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T00:39:52.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:52.086 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:52.101 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-05-27T00:39:52.102 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:52.103 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T00:39:52.103 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:52.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:52.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:52.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:52.239 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T00:39:52.262 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-27T00:39:52.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:52.263 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-05-27T00:39:52.319 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-05-27T00:39:52.319 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T00:39:52.319 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T00:39:52.320 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T00:39:52.320 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T00:39:52.320 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:52.320 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:52.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T00:39:52.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T00:39:52.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T00:39:52.320 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:52.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:52.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:52.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:52.447 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-27T00:39:52.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:52.464 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-05-27T00:39:52.520 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:52.520 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:52.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T00:39:52.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T00:39:52.520 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:52.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:52.631 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:52.637 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T00:39:52.637 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:52.637 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:52.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T00:39:52.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T00:39:52.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T00:39:52.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T00:39:52.638 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T00:39:52.638 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:52.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:52.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:52.750 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:52.750 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:52.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-05-27T00:39:52.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T00:39:52.750 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:52.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:52.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:52.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:52.826 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T00:39:52.841 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T00:39:52.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:52.841 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T00:39:52.896 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T00:39:52.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:52.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:39:52.980 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:39:52.980 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T00:39:52.980 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:39:53.104 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:39:53.104 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T00:39:53.104 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:39:53.104 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:53.105 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:53.109 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:39:53.109 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T00:39:53.109 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:39:53.109 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790" 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790:/ 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=ef97d1dc-a8f7-4487-922a-ccc176f6927a" will pass to kernel 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout: You just mounted a file system that supports labels which does not 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T00:39:53.110 INFO:teuthology.orchestra.run.smithi111.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T00:39:53.111 INFO:teuthology.orchestra.run.smithi111.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T00:39:53.112 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T00:39:53.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:53.112 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T00:39:53.177 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:53.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:53.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:53.276 INFO:teuthology.orchestra.run.smithi111.stdout:client14495 2024-05-27T00:39:53.276 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:39:53.276 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:39:53.293 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:53.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:53.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:53.389 INFO:teuthology.orchestra.run.smithi111.stdout:client14495 2024-05-27T00:39:53.390 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:53.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:53.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:53.446 INFO:teuthology.orchestra.run.smithi111.stdout:client14495 2024-05-27T00:39:53.446 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ef97d1dc-a8f7-4487-922a-ccc176f6927a.client14495/status') 2024-05-27T00:39:53.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:53.504 INFO:teuthology.orchestra.run.smithi111.stdout:instance: client.14495 (3)172.21.15.111:0/3465593960 2024-05-27T00:39:53.504 INFO:teuthology.orchestra.run.smithi111.stdout:blocklisted: false 2024-05-27T00:39:53.505 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records in 2024-05-27T00:39:53.505 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records out 2024-05-27T00:39:53.505 INFO:teuthology.orchestra.run.smithi111.stderr:72 bytes copied, 2.6175e-05 s, 2.8 MB/s 2024-05-27T00:39:53.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:53.511 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-27T00:39:53.512 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T00:39:53.512 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-27T00:39:53.512 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T00:39:53.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.613+0000 7f05492f5640 1 Processor -- start 2024-05-27T00:39:53.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.613+0000 7f05492f5640 1 -- start start 2024-05-27T00:39:53.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05492f5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05440b3630 0x7f05440b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05492f5640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f05440b77e0 0x7f0544161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05492f5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0544162390 0x7f05440d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05492f5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f054406cc70 con 0x7f05440b3630 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05492f5640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f05440d1e50 con 0x7f05440b77e0 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05492f5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f05440d2130 con 0x7f0544162390 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05440b3630 0x7f05440b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f05440b3630 0x7f05440b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45912/0 (socket says 172.21.15.111:45912) 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 -- 172.21.15.111:0/2077022851 learned_addr learned my addr 172.21.15.111:0/2077022851 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05437fe640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0544162390 0x7f05440d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 -- 172.21.15.111:0/2077022851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0544162390 msgr2=0x7f05440d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0544162390 0x7f05440d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 -- 172.21.15.111:0/2077022851 >> v2:172.21.15.153:3300/0 conn(0x7f05440b77e0 msgr2=0x7f0544161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 --2- 172.21.15.111:0/2077022851 >> v2:172.21.15.153:3300/0 conn(0x7f05440b77e0 0x7f0544161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f0542ffd640 1 -- 172.21.15.111:0/2077022851 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05440d2410 con 0x7f05440b3630 2024-05-27T00:39:53.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.614+0000 7f05437fe640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0544162390 0x7f05440d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:53.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.615+0000 7f0542ffd640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05440b3630 0x7f05440b72a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f054406d5d0 tx=0x7f053402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dda043df132bd42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:53.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.615+0000 7f0541ffb640 1 -- 172.21.15.111:0/2077022851 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534037070 con 0x7f05440b3630 2024-05-27T00:39:53.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.615+0000 7f0541ffb640 1 -- 172.21.15.111:0/2077022851 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0534032440 con 0x7f05440b3630 2024-05-27T00:39:53.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.615+0000 7f0541ffb640 1 -- 172.21.15.111:0/2077022851 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534032810 con 0x7f05440b3630 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 -- 172.21.15.111:0/2077022851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05440b3630 msgr2=0x7f05440b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05440b3630 0x7f05440b72a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f054406d5d0 tx=0x7f053402a1e0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 -- 172.21.15.111:0/2077022851 shutdown_connections 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0544162390 0x7f05440d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 --2- 172.21.15.111:0/2077022851 >> v2:172.21.15.153:3300/0 conn(0x7f05440b77e0 0x7f0544161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 --2- 172.21.15.111:0/2077022851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05440b3630 0x7f05440b72a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 -- 172.21.15.111:0/2077022851 >> 172.21.15.111:0/2077022851 conn(0x7f05440d42c0 msgr2=0x7f05440d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 -- 172.21.15.111:0/2077022851 shutdown_connections 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.616+0000 7f05492f5640 1 -- 172.21.15.111:0/2077022851 wait complete. 2024-05-27T00:39:53.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.617+0000 7f05492f5640 1 Processor -- start 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.617+0000 7f05492f5640 1 -- start start 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05492f5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05440b3630 0x7f0544161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05492f5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05440b77e0 0x7f05440b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05492f5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 0x7f05440c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05492f5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f05440d32d0 con 0x7f0544162390 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05492f5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f05440d3150 con 0x7f05440b3630 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05492f5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f05440d3450 con 0x7f05440b77e0 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 0x7f05440c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 0x7f05440c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45926/0 (socket says 172.21.15.111:45926) 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 -- 172.21.15.111:0/2105067139 learned_addr learned my addr 172.21.15.111:0/2105067139 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05427fc640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05440b77e0 0x7f05440b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:53.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f0542ffd640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05440b3630 0x7f0544161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 -- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05440b77e0 msgr2=0x7f05440b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05440b77e0 0x7f05440b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 -- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05440b3630 msgr2=0x7f0544161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05440b3630 0x7f0544161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.618+0000 7f05437fe640 1 -- 172.21.15.111:0/2105067139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05440b05e0 con 0x7f0544162390 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.619+0000 7f05437fe640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 0x7f05440c7bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f053c00cef0 tx=0x7f053c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.619+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c01a070 con 0x7f0544162390 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.619+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f053c00d860 con 0x7f0544162390 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.619+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c00dc50 con 0x7f0544162390 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.619+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05441619d0 con 0x7f0544162390 2024-05-27T00:39:53.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.619+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05441f4880 con 0x7f0544162390 2024-05-27T00:39:53.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.620+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f053c01d070 con 0x7f0544162390 2024-05-27T00:39:53.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.620+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05440d2ef0 con 0x7f0544162390 2024-05-27T00:39:53.624 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.620+0000 7f052b7fe640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f053006be30 0x7f053006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:53.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.621+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f053c08b940 con 0x7f0544162390 2024-05-27T00:39:53.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.621+0000 7f0542ffd640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f053006be30 0x7f053006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:53.625 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.621+0000 7f0542ffd640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f053006be30 0x7f053006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0534034000 tx=0x7f053402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:53.626 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.625+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05440d2ef0 con 0x7f0544162390 2024-05-27T00:39:53.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.837+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f05440cef10 con 0x7f0544162390 2024-05-27T00:39:53.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.838+0000 7f052b7fe640 1 -- 172.21.15.111:0/2105067139 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f053c059ab0 con 0x7f0544162390 2024-05-27T00:39:53.839 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.839+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f053006be30 msgr2=0x7f053006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.839+0000 7f05492f5640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f053006be30 0x7f053006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0534034000 tx=0x7f053402e000 comp rx=0 tx=0).stop 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.839+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 msgr2=0x7f05440c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.839+0000 7f05492f5640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 0x7f05440c7bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f053c00cef0 tx=0x7f053c00a710 comp rx=0 tx=0).stop 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 shutdown_connections 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f053006be30 0x7f053006e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0544162390 0x7f05440c7bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f05440b77e0 0x7f05440b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 --2- 172.21.15.111:0/2105067139 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05440b3630 0x7f0544161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 >> 172.21.15.111:0/2105067139 conn(0x7f05440d42c0 msgr2=0x7f05440b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 shutdown_connections 2024-05-27T00:39:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:53.840+0000 7f05492f5640 1 -- 172.21.15.111:0/2105067139 wait complete. 2024-05-27T00:39:53.851 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T00:39:53.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T00:39:53.852 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-27T00:39:53.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:53.853 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-05-27T00:39:53.870 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:53.871 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42726sec preferred_lft 42726sec 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether fa:19:15:ca:eb:c8 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:53.872 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::50a1:3bff:fe88:d7a2/64 scope link 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether fa:19:15:ca:eb:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::f819:15ff:feca:ebc8/64 scope link tentative 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:53.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:53.873 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T00:39:53.934 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-27T00:39:53.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:53.936 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-05-27T00:39:53.992 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-05-27T00:39:53.992 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T00:39:53.992 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T00:39:53.992 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T00:39:53.992 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T00:39:53.993 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:53.993 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:53.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T00:39:53.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T00:39:53.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T00:39:53.993 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:54.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:54.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:54.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:54.119 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-27T00:39:54.134 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T00:39:54.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:54.135 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-05-27T00:39:54.190 INFO:teuthology.orchestra.run.smithi111.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-27T00:39:54.190 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:54.190 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:54.190 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-27T00:39:54.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-27T00:39:54.191 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:54.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:54.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:54.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:54.306 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-27T00:39:54.330 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:54.330 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:54.330 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 7a:2b:12:3a:b7:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::782b:12ff:fe3a:b7af/64 scope link tentative 2024-05-27T00:39:54.331 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:54.332 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 7a:2b:12:3a:b7:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T00:39:54.397 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T00:39:54.398 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.398 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::782b:12ff:fe3a:b7af/64 scope link tentative 2024-05-27T00:39:54.398 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:54.399 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-27T00:39:54.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-27T00:39:54.400 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:54.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:54.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:54.552 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:54.552 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:39:54.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.1 up 2024-05-27T00:39:54.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.1 master ceph-brx 2024-05-27T00:39:54.552 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:39:54.600 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:54.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:54.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:54.628 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-27T00:39:54.656 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-27T00:39:54.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:54.657 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-27T00:39:54.712 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T00:39:54.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:54.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-27T00:39:54.795 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-27T00:39:54.795 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-27T00:39:54.795 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:39:54.804 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-27T00:39:54.804 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-27T00:39:54.804 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:39:54.804 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:54.804 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790" 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790:/ 2024-05-27T00:39:54.809 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=ef97d1dc-a8f7-4487-922a-ccc176f6927a" will pass to kernel 2024-05-27T00:39:54.810 INFO:teuthology.orchestra.run.smithi111.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-27T00:39:54.810 INFO:teuthology.orchestra.run.smithi111.stdout: You just mounted a file system that supports labels which does not 2024-05-27T00:39:54.810 INFO:teuthology.orchestra.run.smithi111.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T00:39:54.810 INFO:teuthology.orchestra.run.smithi111.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T00:39:54.810 INFO:teuthology.orchestra.run.smithi111.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T00:39:54.811 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T00:39:54.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:54.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-27T00:39:54.876 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:54.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:54.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:54.973 INFO:teuthology.orchestra.run.smithi111.stdout:client4926 2024-05-27T00:39:54.974 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:39:54.974 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:39:54.990 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:55.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:55.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:55.085 INFO:teuthology.orchestra.run.smithi111.stdout:client4926 2024-05-27T00:39:55.085 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:55.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:55.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:55.141 INFO:teuthology.orchestra.run.smithi111.stdout:client4926 2024-05-27T00:39:55.142 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ef97d1dc-a8f7-4487-922a-ccc176f6927a.client4926/status') 2024-05-27T00:39:55.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:55.200 INFO:teuthology.orchestra.run.smithi111.stdout:instance: client.4926 (3)192.168.0.2:0/309030132 2024-05-27T00:39:55.200 INFO:teuthology.orchestra.run.smithi111.stdout:blocklisted: false 2024-05-27T00:39:55.201 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records in 2024-05-27T00:39:55.201 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records out 2024-05-27T00:39:55.201 INFO:teuthology.orchestra.run.smithi111.stderr:68 bytes copied, 2.1941e-05 s, 3.1 MB/s 2024-05-27T00:39:55.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:55.207 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-27T00:39:55.208 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T00:39:55.208 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-27T00:39:55.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T00:39:55.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.306+0000 7fc8c822a640 1 Processor -- start 2024-05-27T00:39:55.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.306+0000 7fc8c822a640 1 -- start start 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c822a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc8c013ef60 0x7fc8c00cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c822a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c00ccfd0 0x7fc8c0141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c822a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c0142650 con 0x7fc8c013f340 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c822a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c00cf210 con 0x7fc8c013ef60 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c822a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c00cebc0 con 0x7fc8c00ccfd0 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c5fc6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc8c013ef60 0x7fc8c00cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:55.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c57c5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc8c013f340 0x7fc8c00cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59048/0 (socket says 172.21.15.111:59048) 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c57c5640 1 -- 172.21.15.111:0/4011640184 learned_addr learned my addr 172.21.15.111:0/4011640184 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c67c7640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c00ccfd0 0x7fc8c0141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c5fc6640 1 -- 172.21.15.111:0/4011640184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c00ccfd0 msgr2=0x7fc8c0141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c5fc6640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c00ccfd0 0x7fc8c0141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c5fc6640 1 -- 172.21.15.111:0/4011640184 >> v1:172.21.15.111:6789/0 conn(0x7fc8c013f340 legacy=0x7fc8c00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c4fc4640 1 -- 172.21.15.111:0/4011640184 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989153652 0 0) 0x7fc8c0142650 con 0x7fc8c013f340 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c5fc6640 1 -- 172.21.15.111:0/4011640184 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c00ceea0 con 0x7fc8c013ef60 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.307+0000 7fc8c67c7640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c00ccfd0 0x7fc8c0141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.308+0000 7fc8c5fc6640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013ef60 0x7fc8c00cf6a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009d90 tx=0x7fc8bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b611e4cbc949768 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.308+0000 7fc8c4fc4640 1 -- 172.21.15.111:0/4011640184 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007740 con 0x7fc8c013ef60 2024-05-27T00:39:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.308+0000 7fc8c4fc4640 1 -- 172.21.15.111:0/4011640184 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8bc0078e0 con 0x7fc8c013ef60 2024-05-27T00:39:55.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.308+0000 7fc8c4fc4640 1 -- 172.21.15.111:0/4011640184 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc007cb0 con 0x7fc8c013ef60 2024-05-27T00:39:55.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 -- 172.21.15.111:0/4011640184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013ef60 msgr2=0x7fc8c00cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:55.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013ef60 0x7fc8c00cf6a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009d90 tx=0x7fc8bc02a660 comp rx=0 tx=0).stop 2024-05-27T00:39:55.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 -- 172.21.15.111:0/4011640184 shutdown_connections 2024-05-27T00:39:55.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c00ccfd0 0x7fc8c0141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 --2- 172.21.15.111:0/4011640184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013ef60 0x7fc8c00cf6a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 -- 172.21.15.111:0/4011640184 >> 172.21.15.111:0/4011640184 conn(0x7fc8c00b1c00 msgr2=0x7fc8c00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 -- 172.21.15.111:0/4011640184 shutdown_connections 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.309+0000 7fc8c822a640 1 -- 172.21.15.111:0/4011640184 wait complete. 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.310+0000 7fc8c822a640 1 Processor -- start 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.310+0000 7fc8c822a640 1 -- start start 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c00ccfd0 0x7fc8c0140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c013ef60 0x7fc8c01401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 0x7fc8c013e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0141e60 con 0x7fc8c013f340 2024-05-27T00:39:55.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c00ce890 con 0x7fc8c00ccfd0 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c0141fe0 con 0x7fc8c013ef60 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 0x7fc8c013e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 0x7fc8c013e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45944/0 (socket says 172.21.15.111:45944) 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 -- 172.21.15.111:0/2274589604 learned_addr learned my addr 172.21.15.111:0/2274589604 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c57c5640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c013ef60 0x7fc8c01401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c5fc6640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c00ccfd0 0x7fc8c0140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 -- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c013ef60 msgr2=0x7fc8c01401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c013ef60 0x7fc8c01401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 -- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c00ccfd0 msgr2=0x7fc8c0140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c00ccfd0 0x7fc8c0140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 -- 172.21.15.111:0/2274589604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c0141a40 con 0x7fc8c013f340 2024-05-27T00:39:55.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c67c7640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 0x7fc8c013e7b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c7f0 tx=0x7fc8b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8015070 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8007820 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c0141480 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8007c10 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.311+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c0142ad0 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.312+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c0068680 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.313+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8b8018070 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.313+0000 7fc8b67fc640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc8a406be30 0x7fc8a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.313+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc8b808b190 con 0x7fc8c013f340 2024-05-27T00:39:55.317 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.314+0000 7fc8c5fc6640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc8a406be30 0x7fc8a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:55.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.314+0000 7fc8c5fc6640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc8a406be30 0x7fc8a406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc034000 tx=0x7fc8bc02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:55.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.317+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc8b80591b0 con 0x7fc8c013f340 2024-05-27T00:39:55.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.526+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8c01606a0 con 0x7fc8c013f340 2024-05-27T00:39:55.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.527+0000 7fc8b67fc640 1 -- 172.21.15.111:0/2274589604 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fc8b805d4b0 con 0x7fc8c013f340 2024-05-27T00:39:55.528 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T00:39:55.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc8a406be30 msgr2=0x7fc8a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:55.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc8a406be30 0x7fc8a406e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc034000 tx=0x7fc8bc02f000 comp rx=0 tx=0).stop 2024-05-27T00:39:55.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 msgr2=0x7fc8c013e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 0x7fc8c013e7b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c7f0 tx=0x7fc8b800ccb0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 shutdown_connections 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc8a406be30 0x7fc8a406e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8c013f340 0x7fc8c013e7b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc8c013ef60 0x7fc8c01401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 --2- 172.21.15.111:0/2274589604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c00ccfd0 0x7fc8c0140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 >> 172.21.15.111:0/2274589604 conn(0x7fc8c00b1c00 msgr2=0x7fc8c00afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 shutdown_connections 2024-05-27T00:39:55.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:55.529+0000 7fc8c822a640 1 -- 172.21.15.111:0/2274589604 wait complete. 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T00:39:55.541 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-27T00:39:55.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:55.542 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:55.561 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2024-05-27T00:39:55.562 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:55.563 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add name ceph-brx type bridge 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr flush dev ceph-brx 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set ceph-brx up 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T00:39:55.563 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:55.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:55.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:55.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:55.777 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T00:39:55.801 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-05-27T00:39:55.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:55.803 DEBUG:teuthology.orchestra.run.smithi153:> route 2024-05-27T00:39:55.877 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2024-05-27T00:39:55.877 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T00:39:55.877 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T00:39:55.877 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T00:39:55.877 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T00:39:55.878 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:55.878 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:55.878 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T00:39:55.878 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T00:39:55.878 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T00:39:55.878 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:55.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:56.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:56.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:56.176 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2024-05-27T00:39:56.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:56.196 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2024-05-27T00:39:56.252 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:56.252 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:56.252 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-27T00:39:56.252 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-27T00:39:56.252 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:56.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:56.372 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-27T00:39:56.372 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:56.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:56.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:56.504 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:56.504 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:56.505 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2024-05-27T00:39:56.505 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T00:39:56.505 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:56.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:56.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:56.582 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-27T00:39:56.597 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-27T00:39:56.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:56.597 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-27T00:39:56.653 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T00:39:56.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:56.653 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T00:39:56.747 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T00:39:56.748 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-27T00:39:56.748 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2024-05-27T00:39:56.872 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T00:39:56.872 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-27T00:39:56.872 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2024-05-27T00:39:56.872 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:56.872 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790" 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790:/ 2024-05-27T00:39:56.881 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=ef97d1dc-a8f7-4487-922a-ccc176f6927a" will pass to kernel 2024-05-27T00:39:56.882 INFO:teuthology.orchestra.run.smithi153.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-27T00:39:56.882 INFO:teuthology.orchestra.run.smithi153.stdout: You just mounted a file system that supports labels which does not 2024-05-27T00:39:56.882 INFO:teuthology.orchestra.run.smithi153.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T00:39:56.882 INFO:teuthology.orchestra.run.smithi153.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T00:39:56.882 INFO:teuthology.orchestra.run.smithi153.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T00:39:56.884 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T00:39:56.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:56.884 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-27T00:39:56.909 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:57.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:57.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:57.007 INFO:teuthology.orchestra.run.smithi153.stdout:client4732 2024-05-27T00:39:57.007 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:39:57.008 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:39:57.024 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:57.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:57.122 INFO:teuthology.orchestra.run.smithi153.stdout:client4732 2024-05-27T00:39:57.122 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:57.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:57.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:57.178 INFO:teuthology.orchestra.run.smithi153.stdout:client4732 2024-05-27T00:39:57.179 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ef97d1dc-a8f7-4487-922a-ccc176f6927a.client4732/status') 2024-05-27T00:39:57.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:57.238 INFO:teuthology.orchestra.run.smithi153.stdout:instance: client.4732 (3)192.168.0.1:0/598050121 2024-05-27T00:39:57.238 INFO:teuthology.orchestra.run.smithi153.stdout:blocklisted: false 2024-05-27T00:39:57.238 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records in 2024-05-27T00:39:57.238 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records out 2024-05-27T00:39:57.238 INFO:teuthology.orchestra.run.smithi153.stderr:68 bytes copied, 2.0937e-05 s, 3.2 MB/s 2024-05-27T00:39:57.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:57.245 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-27T00:39:57.245 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T00:39:57.245 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-27T00:39:57.245 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T00:39:57.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.355+0000 7faa88189640 1 Processor -- start 2024-05-27T00:39:57.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.355+0000 7faa88189640 1 -- start start 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa88189640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faa8013ef60 0x7faa800cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa88189640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa8013f340 0x7faa800cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa88189640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa80142650 con 0x7faa800cce00 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa88189640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7faa800cf210 con 0x7faa8013ef60 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa88189640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faa800cebc0 con 0x7faa8013f340 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa86726640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7faa800cce00 0x7faa80141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59052/0 (socket says 172.21.15.111:59052) 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.356+0000 7faa86726640 1 -- 172.21.15.111:0/3659302828 learned_addr learned my addr 172.21.15.111:0/3659302828 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85724640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa8013f340 0x7faa800cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85f25640 1 --2- 172.21.15.111:0/3659302828 >> v2:172.21.15.153:3300/0 conn(0x7faa8013ef60 0x7faa800cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa84f23640 1 -- 172.21.15.111:0/3659302828 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284166173 0 0) 0x7faa80142650 con 0x7faa800cce00 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa84f23640 1 -- 172.21.15.111:0/3659302828 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa68003520 con 0x7faa800cce00 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85f25640 1 -- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa8013f340 msgr2=0x7faa800cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85f25640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa8013f340 0x7faa800cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85f25640 1 -- 172.21.15.111:0/3659302828 >> v1:172.21.15.111:6789/0 conn(0x7faa800cce00 legacy=0x7faa80141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85f25640 1 -- 172.21.15.111:0/3659302828 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa800ceea0 con 0x7faa8013ef60 2024-05-27T00:39:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.357+0000 7faa85724640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa8013f340 0x7faa800cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa85f25640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa800cf6a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa70009d90 tx=0x7faa7002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7f3a804db97829 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa84f23640 1 -- 172.21.15.111:0/3659302828 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70007740 con 0x7faa8013ef60 2024-05-27T00:39:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa84f23640 1 -- 172.21.15.111:0/3659302828 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa700078e0 con 0x7faa8013ef60 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa84f23640 1 -- 172.21.15.111:0/3659302828 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70007c90 con 0x7faa8013ef60 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa88189640 1 -- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 msgr2=0x7faa800cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa88189640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa800cf6a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa70009d90 tx=0x7faa7002a660 comp rx=0 tx=0).stop 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa88189640 1 -- 172.21.15.111:0/3659302828 shutdown_connections 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa88189640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa8013f340 0x7faa800cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa88189640 1 --2- 172.21.15.111:0/3659302828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa800cf6a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.358+0000 7faa88189640 1 -- 172.21.15.111:0/3659302828 >> 172.21.15.111:0/3659302828 conn(0x7faa800b1c00 msgr2=0x7faa800b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.359+0000 7faa88189640 1 -- 172.21.15.111:0/3659302828 shutdown_connections 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.359+0000 7faa88189640 1 -- 172.21.15.111:0/3659302828 wait complete. 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 Processor -- start 2024-05-27T00:39:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 -- start start 2024-05-27T00:39:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa800cce00 0x7faa80158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa80157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 0x7faa801408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa80142030 con 0x7faa8013f340 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7faa800ce890 con 0x7faa8013ef60 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.360+0000 7faa88189640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7faa801421b0 con 0x7faa800cce00 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 0x7faa801408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 0x7faa801408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45958/0 (socket says 172.21.15.111:45958) 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 -- 172.21.15.111:0/2824571675 learned_addr learned my addr 172.21.15.111:0/2824571675 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa85724640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa80157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa85f25640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa800cce00 0x7faa80158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 -- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa800cce00 msgr2=0x7faa80158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa800cce00 0x7faa80158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 -- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 msgr2=0x7faa80157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa80157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 -- 172.21.15.111:0/2824571675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa801f1b00 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa85724640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa80157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa86726640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 0x7faa801408a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c430 tx=0x7faa7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78017070 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa78005ce0 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa801f5550 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78007310 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.361+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801592b0 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.362+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa80068680 con 0x7faa8013f340 2024-05-27T00:39:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.363+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa7801a070 con 0x7faa8013f340 2024-05-27T00:39:57.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.363+0000 7faa7e7fc640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faa6c06be30 0x7faa6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:57.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.363+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa7808a580 con 0x7faa8013f340 2024-05-27T00:39:57.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.363+0000 7faa85f25640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faa6c06be30 0x7faa6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:57.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.364+0000 7faa85f25640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faa6c06be30 0x7faa6c06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa70034000 tx=0x7faa7002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.368+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faa78058630 con 0x7faa8013f340 2024-05-27T00:39:57.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.579+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faa80145a40 con 0x7faa8013f340 2024-05-27T00:39:57.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.579+0000 7faa7e7fc640 1 -- 172.21.15.111:0/2824571675 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7faa7805c930 con 0x7faa8013f340 2024-05-27T00:39:57.581 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T00:39:57.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faa6c06be30 msgr2=0x7faa6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faa6c06be30 0x7faa6c06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa70034000 tx=0x7faa7002f000 comp rx=0 tx=0).stop 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 msgr2=0x7faa801408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 0x7faa801408a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c430 tx=0x7faa7800c8f0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 shutdown_connections 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7faa6c06be30 0x7faa6c06e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa8013f340 0x7faa801408a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8013ef60 0x7faa80157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 --2- 172.21.15.111:0/2824571675 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7faa800cce00 0x7faa80158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 >> 172.21.15.111:0/2824571675 conn(0x7faa800b1c00 msgr2=0x7faa8006c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:57.583 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 shutdown_connections 2024-05-27T00:39:57.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:57.581+0000 7faa88189640 1 -- 172.21.15.111:0/2824571675 wait complete. 2024-05-27T00:39:57.593 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T00:39:57.593 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T00:39:57.593 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2024-05-27T00:39:57.593 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-27T00:39:57.593 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-27T00:39:57.593 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T00:39:57.594 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T00:39:57.594 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T00:39:57.594 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-27T00:39:57.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:57.594 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2024-05-27T00:39:57.610 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:57.610 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:57.610 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:57.610 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:57.611 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42707sec preferred_lft 42707sec 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 96:e2:e0:3d:bb:03 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::34dc:e9ff:fe49:6c88/64 scope link 2024-05-27T00:39:57.612 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:57.613 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T00:39:57.613 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 96:e2:e0:3d:bb:03 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-27T00:39:57.613 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::94e2:e0ff:fe3d:bb03/64 scope link tentative 2024-05-27T00:39:57.613 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:57.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:57.613 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T00:39:57.674 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-05-27T00:39:57.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:57.676 DEBUG:teuthology.orchestra.run.smithi153:> route 2024-05-27T00:39:57.733 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2024-05-27T00:39:57.734 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T00:39:57.734 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T00:39:57.734 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T00:39:57.734 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T00:39:57.735 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:57.735 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:57.735 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T00:39:57.735 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T00:39:57.735 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T00:39:57.736 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:57.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:57.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:57.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:57.861 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2024-05-27T00:39:57.876 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-27T00:39:57.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:57.877 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2024-05-27T00:39:57.932 INFO:teuthology.orchestra.run.smithi153.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-27T00:39:57.932 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:57.932 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:57.932 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-27T00:39:57.933 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-27T00:39:57.933 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:58.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:58.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:58.052 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-27T00:39:58.076 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:58.076 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:58.076 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 92:a8:1b:1c:c6:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::90a8:1bff:fe1c:c6b5/64 scope link tentative 2024-05-27T00:39:58.077 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:58.078 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-27T00:39:58.143 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 92:a8:1b:1c:c6:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-27T00:39:58.144 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-27T00:39:58.145 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.145 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::90a8:1bff:fe1c:c6b5/64 scope link tentative 2024-05-27T00:39:58.145 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:39:58.146 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-27T00:39:58.146 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:58.146 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:58.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-27T00:39:58.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-27T00:39:58.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-27T00:39:58.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-27T00:39:58.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-27T00:39:58.147 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:58.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:58.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.299 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:39:58.299 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-05-27T00:39:58.300 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.1 up 2024-05-27T00:39:58.300 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.1 master ceph-brx 2024-05-27T00:39:58.300 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-05-27T00:39:58.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:58.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:58.378 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-27T00:39:58.394 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-27T00:39:58.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:39:58.394 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-27T00:39:58.449 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T00:39:58.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:58.449 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-27T00:39:58.532 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-27T00:39:58.533 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-27T00:39:58.533 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2024-05-27T00:39:58.541 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-27T00:39:58.541 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-27T00:39:58.541 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2024-05-27T00:39:58.541 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:58.541 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790" 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790:/ 2024-05-27T00:39:58.546 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=ef97d1dc-a8f7-4487-922a-ccc176f6927a" will pass to kernel 2024-05-27T00:39:58.547 INFO:teuthology.orchestra.run.smithi153.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-27T00:39:58.547 INFO:teuthology.orchestra.run.smithi153.stdout: You just mounted a file system that supports labels which does not 2024-05-27T00:39:58.547 INFO:teuthology.orchestra.run.smithi153.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T00:39:58.547 INFO:teuthology.orchestra.run.smithi153.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T00:39:58.547 INFO:teuthology.orchestra.run.smithi153.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T00:39:58.548 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T00:39:58.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:39:58.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-27T00:39:58.614 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:58.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:58.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.712 INFO:teuthology.orchestra.run.smithi153.stdout:client14510 2024-05-27T00:39:58.712 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:39:58.713 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:39:58.729 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:58.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:58.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.825 INFO:teuthology.orchestra.run.smithi153.stdout:client14510 2024-05-27T00:39:58.825 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:39:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:39:58.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.881 INFO:teuthology.orchestra.run.smithi153.stdout:client14510 2024-05-27T00:39:58.882 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ef97d1dc-a8f7-4487-922a-ccc176f6927a.client14510/status') 2024-05-27T00:39:58.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:39:58.940 INFO:teuthology.orchestra.run.smithi153.stdout:instance: client.14510 (3)192.168.0.2:0/2500147129 2024-05-27T00:39:58.940 INFO:teuthology.orchestra.run.smithi153.stdout:blocklisted: false 2024-05-27T00:39:58.940 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records in 2024-05-27T00:39:58.940 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records out 2024-05-27T00:39:58.940 INFO:teuthology.orchestra.run.smithi153.stderr:70 bytes copied, 2.5063e-05 s, 2.8 MB/s 2024-05-27T00:39:58.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-05-27T00:39:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:39:58.948 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-27T00:39:58.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:39:59.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.077+0000 7f822f0b2640 1 Processor -- start 2024-05-27T00:39:59.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.077+0000 7f822f0b2640 1 -- start start 2024-05-27T00:39:59.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822f0b2640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8228144000 0x7f82280b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822f0b2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82280726d0 0x7f8228072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822f0b2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8228073020 con 0x7f822806b390 2024-05-27T00:39:59.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822f0b2640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f82280b42f0 con 0x7f8228144000 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822f0b2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f82280b3440 con 0x7f82280726d0 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f8227fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f822806b390 0x7f82280cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59066/0 (socket says 172.21.15.111:59066) 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f8227fff640 1 -- 172.21.15.111:0/2811694770 learned_addr learned my addr 172.21.15.111:0/2811694770 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822d64f640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82280726d0 0x7f8228072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822ce4e640 1 --2- 172.21.15.111:0/2811694770 >> v2:172.21.15.153:3300/0 conn(0x7f8228144000 0x7f82280b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f82277fe640 1 -- 172.21.15.111:0/2811694770 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1837120855 0 0) 0x7f8214001110 con 0x7f822806b390 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f82277fe640 1 -- 172.21.15.111:0/2811694770 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8218003520 con 0x7f822806b390 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822ce4e640 1 -- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82280726d0 msgr2=0x7f8228072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822ce4e640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82280726d0 0x7f8228072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822ce4e640 1 -- 172.21.15.111:0/2811694770 >> v1:172.21.15.111:6789/0 conn(0x7f822806b390 legacy=0x7f82280cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822ce4e640 1 -- 172.21.15.111:0/2811694770 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280b36e0 con 0x7f8228144000 2024-05-27T00:39:59.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.078+0000 7f822d64f640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82280726d0 0x7f8228072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.079+0000 7f822ce4e640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228144000 0x7f82280b2f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f821c0097a0 tx=0x7f821c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab565958b6c415b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.079+0000 7f82277fe640 1 -- 172.21.15.111:0/2811694770 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c037070 con 0x7f8228144000 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.079+0000 7f82277fe640 1 -- 172.21.15.111:0/2811694770 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821c032440 con 0x7f8228144000 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.079+0000 7f82277fe640 1 -- 172.21.15.111:0/2811694770 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0327f0 con 0x7f8228144000 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 -- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228144000 msgr2=0x7f82280b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228144000 0x7f82280b2f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f821c0097a0 tx=0x7f821c02a180 comp rx=0 tx=0).stop 2024-05-27T00:39:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 -- 172.21.15.111:0/2811694770 shutdown_connections 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f82280726d0 0x7f8228072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 --2- 172.21.15.111:0/2811694770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8228144000 0x7f82280b2f00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 -- 172.21.15.111:0/2811694770 >> 172.21.15.111:0/2811694770 conn(0x7f82280b1c00 msgr2=0x7f82280b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 -- 172.21.15.111:0/2811694770 shutdown_connections 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.080+0000 7f822f0b2640 1 -- 172.21.15.111:0/2811694770 wait complete. 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.081+0000 7f822f0b2640 1 Processor -- start 2024-05-27T00:39:59.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.081+0000 7f822f0b2640 1 -- start start 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822f0b2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f822806b390 0x7f822815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822f0b2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 0x7f822813e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822f0b2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8228144000 0x7f82280ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822f0b2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f82280b3f80 con 0x7f82280726d0 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822f0b2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f82280b3ca0 con 0x7f822806b390 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822f0b2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f82280cde50 con 0x7f8228144000 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 0x7f822813e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 0x7f822813e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45972/0 (socket says 172.21.15.111:45972) 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 -- 172.21.15.111:0/1852292928 learned_addr learned my addr 172.21.15.111:0/1852292928 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822ce4e640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f822806b390 0x7f822815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822d64f640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8228144000 0x7f82280ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 -- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8228144000 msgr2=0x7f82280ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8228144000 0x7f82280ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 -- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f822806b390 msgr2=0x7f822815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f822806b390 0x7f822815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f8227fff640 1 -- 172.21.15.111:0/1852292928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280c5810 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.082+0000 7f822ce4e640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f822806b390 0x7f822815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.083+0000 7f8227fff640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 0x7f822813e920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f821400c3c0 tx=0x7f821400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.083+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214017070 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.083+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8214005ce0 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.083+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82280c5ad0 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.083+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140072f0 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.083+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82280ab790 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.084+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8228068680 con 0x7f82280726d0 2024-05-27T00:39:59.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.084+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f821401a070 con 0x7f82280726d0 2024-05-27T00:39:59.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.085+0000 7f82257fa640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f820c06be30 0x7f820c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.085+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f821408a370 con 0x7f82280726d0 2024-05-27T00:39:59.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.085+0000 7f822ce4e640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f820c06be30 0x7f820c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.086+0000 7f822ce4e640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f820c06be30 0x7f820c06e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f821c034000 tx=0x7f821c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.089+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8228073020 con 0x7f82280726d0 2024-05-27T00:39:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.267+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82280715e0 con 0x7f82280726d0 2024-05-27T00:39:59.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.268+0000 7f82257fa640 1 -- 172.21.15.111:0/1852292928 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f8214058390 con 0x7f82280726d0 2024-05-27T00:39:59.270 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:39:59.270 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:39:59.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f820c06be30 msgr2=0x7f820c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f820c06be30 0x7f820c06e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f821c034000 tx=0x7f821c02e000 comp rx=0 tx=0).stop 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 msgr2=0x7f822813e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 0x7f822813e920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f821400c3c0 tx=0x7f821400c7c0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 shutdown_connections 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f820c06be30 0x7f820c06e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8228144000 0x7f82280ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.270+0000 7f822f0b2640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82280726d0 0x7f822813e920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.271+0000 7f822f0b2640 1 --2- 172.21.15.111:0/1852292928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f822806b390 0x7f822815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.271+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 >> 172.21.15.111:0/1852292928 conn(0x7f82280b1c00 msgr2=0x7f82280ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.271+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 shutdown_connections 2024-05-27T00:39:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.271+0000 7f822f0b2640 1 -- 172.21.15.111:0/1852292928 wait complete. 2024-05-27T00:39:59.283 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:39:59.283 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:39:59.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.387+0000 7f3052ce1640 1 Processor -- start 2024-05-27T00:39:59.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.387+0000 7f3052ce1640 1 -- start start 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3052ce1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0c6e20 0x7f304c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3052ce1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f304c0cf080 0x7f304c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3052ce1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f304c0c72b0 con 0x7f304c147120 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3052ce1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f304c0c7590 con 0x7f304c0cf080 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3052ce1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f304c0c7870 con 0x7f304c0c6e20 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f305127e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f304c0cf080 0x7f304c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3050a7d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f304c147120 0x7f304c0ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59068/0 (socket says 172.21.15.111:59068) 2024-05-27T00:39:59.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f3050a7d640 1 -- 172.21.15.111:0/2967304417 learned_addr learned my addr 172.21.15.111:0/2967304417 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f304bfff640 1 --2- 172.21.15.111:0/2967304417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0c6e20 0x7f304c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f305127e640 1 -- 172.21.15.111:0/2967304417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0c6e20 msgr2=0x7f304c0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f305127e640 1 --2- 172.21.15.111:0/2967304417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0c6e20 0x7f304c0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f305127e640 1 -- 172.21.15.111:0/2967304417 >> v1:172.21.15.111:6789/0 conn(0x7f304c147120 legacy=0x7f304c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.388+0000 7f305127e640 1 -- 172.21.15.111:0/2967304417 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c0c7b50 con 0x7f304c0cf080 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.389+0000 7f305127e640 1 --2- 172.21.15.111:0/2967304417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0cc5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3044009d90 tx=0x7f304402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991bdb2003311fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.389+0000 7f304b7fe640 1 -- 172.21.15.111:0/2967304417 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044007740 con 0x7f304c0cf080 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.389+0000 7f304b7fe640 1 -- 172.21.15.111:0/2967304417 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30440078e0 con 0x7f304c0cf080 2024-05-27T00:39:59.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.389+0000 7f304b7fe640 1 -- 172.21.15.111:0/2967304417 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044007cb0 con 0x7f304c0cf080 2024-05-27T00:39:59.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 -- 172.21.15.111:0/2967304417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 msgr2=0x7f304c0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 --2- 172.21.15.111:0/2967304417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0cc5b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3044009d90 tx=0x7f304402a660 comp rx=0 tx=0).stop 2024-05-27T00:39:59.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 -- 172.21.15.111:0/2967304417 shutdown_connections 2024-05-27T00:39:59.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 --2- 172.21.15.111:0/2967304417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0cc5b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 --2- 172.21.15.111:0/2967304417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0c6e20 0x7f304c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 -- 172.21.15.111:0/2967304417 >> 172.21.15.111:0/2967304417 conn(0x7f304c0d1f60 msgr2=0x7f304c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:59.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 -- 172.21.15.111:0/2967304417 shutdown_connections 2024-05-27T00:39:59.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.390+0000 7f3052ce1640 1 -- 172.21.15.111:0/2967304417 wait complete. 2024-05-27T00:39:59.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.391+0000 7f3052ce1640 1 Processor -- start 2024-05-27T00:39:59.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.391+0000 7f3052ce1640 1 -- start start 2024-05-27T00:39:59.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3052ce1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3052ce1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c147120 0x7f304c0b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3052ce1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0b63d0 0x7f304c0b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3052ce1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1477d0 con 0x7f304c0cf080 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3052ce1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0c8350 con 0x7f304c147120 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3052ce1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f304c147950 con 0x7f304c0b63d0 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45984/0 (socket says 172.21.15.111:45984) 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 -- 172.21.15.111:0/3846603343 learned_addr learned my addr 172.21.15.111:0/3846603343 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f305127e640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0b63d0 0x7f304c0b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f304bfff640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c147120 0x7f304c0b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 -- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0b63d0 msgr2=0x7f304c0b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0b63d0 0x7f304c0b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 -- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c147120 msgr2=0x7f304c0b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c147120 0x7f304c0b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.392+0000 7f3050a7d640 1 -- 172.21.15.111:0/3846603343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c0b5330 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.393+0000 7f3050a7d640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0b4c70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f304000c3c0 tx=0x7f304000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.393+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040017070 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.393+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040005ce0 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.393+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30400072f0 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.393+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c0b5610 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.393+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c0b5bd0 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.394+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c068680 con 0x7f304c0cf080 2024-05-27T00:39:59.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.394+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f304001a070 con 0x7f304c0cf080 2024-05-27T00:39:59.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.395+0000 7f30497fa640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f303006be30 0x7f303006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.395+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f304008a4b0 con 0x7f304c0cf080 2024-05-27T00:39:59.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.395+0000 7f304bfff640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f303006be30 0x7f303006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.395+0000 7f304bfff640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f303006be30 0x7f303006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3038006fd0 tx=0x7f3038008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.399+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3040058bb0 con 0x7f304c0cf080 2024-05-27T00:39:59.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.580+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f304c0b3140 con 0x7f304c0cf080 2024-05-27T00:39:59.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.581+0000 7f30497fa640 1 -- 172.21.15.111:0/3846603343 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f30400584b0 con 0x7f304c0cf080 2024-05-27T00:39:59.582 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f303006be30 msgr2=0x7f303006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f303006be30 0x7f303006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3038006fd0 tx=0x7f3038008040 comp rx=0 tx=0).stop 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 msgr2=0x7f304c0b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0b4c70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f304000c3c0 tx=0x7f304000c880 comp rx=0 tx=0).stop 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 shutdown_connections 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f303006be30 0x7f303006e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f304c0b63d0 0x7f304c0b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c147120 0x7f304c0b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 --2- 172.21.15.111:0/3846603343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f304c0cf080 0x7f304c0b4c70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 >> 172.21.15.111:0/3846603343 conn(0x7f304c0d1f60 msgr2=0x7f304c06ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:59.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 shutdown_connections 2024-05-27T00:39:59.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.583+0000 7f3052ce1640 1 -- 172.21.15.111:0/3846603343 wait complete. 2024-05-27T00:39:59.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:39:59.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.701+0000 7f89bd2cd640 1 Processor -- start 2024-05-27T00:39:59.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.701+0000 7f89bd2cd640 1 -- start start 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.702+0000 7f89bd2cd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813ef60 0x7f89b80cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.702+0000 7f89bd2cd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89b813f340 0x7f89b80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.702+0000 7f89bd2cd640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b8142650 con 0x7f89b80cce00 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.702+0000 7f89bd2cd640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f89b80cf210 con 0x7f89b813f340 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.702+0000 7f89bd2cd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f89b80cebc0 con 0x7f89b813ef60 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b67fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89b813f340 0x7f89b80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b6ffd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813ef60 0x7f89b80cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f89b80cce00 0x7f89b8141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59072/0 (socket says 172.21.15.111:59072) 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b67fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89b813f340 0x7f89b80cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45996/0 (socket says 172.21.15.111:45996) 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b77fe640 1 -- 172.21.15.111:0/2626110711 learned_addr learned my addr 172.21.15.111:0/2626110711 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b67fc640 1 -- 172.21.15.111:0/2626110711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813ef60 msgr2=0x7f89b80cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b67fc640 1 --2- 172.21.15.111:0/2626110711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813ef60 0x7f89b80cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b67fc640 1 -- 172.21.15.111:0/2626110711 >> v1:172.21.15.111:6789/0 conn(0x7f89b80cce00 legacy=0x7f89b8141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b67fc640 1 -- 172.21.15.111:0/2626110711 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80ceea0 con 0x7f89b813f340 2024-05-27T00:39:59.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.703+0000 7f89b5ffb640 1 -- 172.21.15.111:0/2626110711 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1141578570 0 0) 0x7f89b8142650 con 0x7f89b80cce00 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89b67fc640 1 --2- 172.21.15.111:0/2626110711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b813f340 0x7f89b80cc8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009d90 tx=0x7f89ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ba3845f0a27e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89b5ffb640 1 -- 172.21.15.111:0/2626110711 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac007740 con 0x7f89b813f340 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89b5ffb640 1 -- 172.21.15.111:0/2626110711 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac0078e0 con 0x7f89b813f340 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89b5ffb640 1 -- 172.21.15.111:0/2626110711 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac007cb0 con 0x7f89b813f340 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89bd2cd640 1 -- 172.21.15.111:0/2626110711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b813f340 msgr2=0x7f89b80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/2626110711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b813f340 0x7f89b80cc8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009d90 tx=0x7f89ac02a660 comp rx=0 tx=0).stop 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89bd2cd640 1 -- 172.21.15.111:0/2626110711 shutdown_connections 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/2626110711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b813f340 0x7f89b80cc8c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.704+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/2626110711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813ef60 0x7f89b80cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.705+0000 7f89bd2cd640 1 -- 172.21.15.111:0/2626110711 >> 172.21.15.111:0/2626110711 conn(0x7f89b80b1c00 msgr2=0x7f89b80b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:59.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.705+0000 7f89bd2cd640 1 -- 172.21.15.111:0/2626110711 shutdown_connections 2024-05-27T00:39:59.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.705+0000 7f89bd2cd640 1 -- 172.21.15.111:0/2626110711 wait complete. 2024-05-27T00:39:59.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.706+0000 7f89bd2cd640 1 Processor -- start 2024-05-27T00:39:59.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.706+0000 7f89bd2cd640 1 -- start start 2024-05-27T00:39:59.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 0x7f89b80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89b813ef60 0x7f89b81f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813f340 0x7f89b815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8142030 con 0x7f89b80cce00 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89b80ce890 con 0x7f89b813ef60 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f89b81421b0 con 0x7f89b813f340 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 0x7f89b80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 0x7f89b80b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46008/0 (socket says 172.21.15.111:46008) 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 -- 172.21.15.111:0/3469787127 learned_addr learned my addr 172.21.15.111:0/3469787127 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b77fe640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813f340 0x7f89b815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b67fc640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89b813ef60 0x7f89b81f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 -- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813f340 msgr2=0x7f89b815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813f340 0x7f89b815e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 -- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89b813ef60 msgr2=0x7f89b81f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89b813ef60 0x7f89b81f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 -- 172.21.15.111:0/3469787127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b81f4b90 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89b6ffd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 0x7f89b80b48e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89b000e9b0 tx=0x7f89b000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b000c9a0 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b80ad990 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000cd90 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.707+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b80adf50 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.708+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b8075570 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.709+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89b001a070 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.709+0000 7f899f7fe640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89a406be30 0x7f89a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.709+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f89b008b130 con 0x7f89b80cce00 2024-05-27T00:39:59.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.710+0000 7f89b67fc640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89a406be30 0x7f89a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:39:59.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.710+0000 7f89b67fc640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89a406be30 0x7f89a406e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89ac034000 tx=0x7f89ac02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:39:59.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.717+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f89b8075570 con 0x7f89b80cce00 2024-05-27T00:39:59.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.899+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89b8068680 con 0x7f89b80cce00 2024-05-27T00:39:59.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.899+0000 7f899f7fe640 1 -- 172.21.15.111:0/3469787127 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f89b81f4b90 con 0x7f89b80cce00 2024-05-27T00:39:59.900 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:39:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89a406be30 msgr2=0x7f89a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89a406be30 0x7f89a406e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89ac034000 tx=0x7f89ac02f000 comp rx=0 tx=0).stop 2024-05-27T00:39:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 msgr2=0x7f89b80b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 0x7f89b80b48e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89b000e9b0 tx=0x7f89b000ee70 comp rx=0 tx=0).stop 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 shutdown_connections 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89a406be30 0x7f89a406e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89b813f340 0x7f89b815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89b813ef60 0x7f89b81f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 --2- 172.21.15.111:0/3469787127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89b80cce00 0x7f89b80b48e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:39:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 >> 172.21.15.111:0/3469787127 conn(0x7f89b80b1c00 msgr2=0x7f89b80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:39:59.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 shutdown_connections 2024-05-27T00:39:59.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:39:59.901+0000 7f89bd2cd640 1 -- 172.21.15.111:0/3469787127 wait complete. 2024-05-27T00:39:59.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:00.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.018+0000 7f2c50438640 1 Processor -- start 2024-05-27T00:40:00.019 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.018+0000 7f2c50438640 1 -- start start 2024-05-27T00:40:00.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c50438640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2c480b3630 0x7f2c480b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c50438640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2c480b77e0 0x7f2c48161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c50438640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4806cc70 con 0x7f2c48162390 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c50438640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f2c480d1e50 con 0x7f2c480b3630 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c50438640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2c480d2130 con 0x7f2c480b77e0 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2c480b3630 0x7f2c480b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2c480b3630 0x7f2c480b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46016/0 (socket says 172.21.15.111:46016) 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 -- 172.21.15.111:0/3921964329 learned_addr learned my addr 172.21.15.111:0/3921964329 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4d9d3640 1 --2- 172.21.15.111:0/3921964329 >> v2:172.21.15.153:3300/0 conn(0x7f2c480b77e0 0x7f2c48161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 -- 172.21.15.111:0/3921964329 >> v2:172.21.15.153:3300/0 conn(0x7f2c480b77e0 msgr2=0x7f2c48161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 --2- 172.21.15.111:0/3921964329 >> v2:172.21.15.153:3300/0 conn(0x7f2c480b77e0 0x7f2c48161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 -- 172.21.15.111:0/3921964329 >> v1:172.21.15.153:6789/0 conn(0x7f2c48162390 legacy=0x7f2c480d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.019+0000 7f2c4e1d4640 1 -- 172.21.15.111:0/3921964329 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c480d2410 con 0x7f2c480b3630 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.020+0000 7f2c4e1d4640 1 --2- 172.21.15.111:0/3921964329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c480b3630 0x7f2c480b72a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c4806d5d0 tx=0x7f2c4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364880fc4af73978 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.020+0000 7f2c4d1d2640 1 -- 172.21.15.111:0/3921964329 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44036070 con 0x7f2c480b3630 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.020+0000 7f2c4d1d2640 1 -- 172.21.15.111:0/3921964329 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c44031600 con 0x7f2c480b3630 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.020+0000 7f2c4d1d2640 1 -- 172.21.15.111:0/3921964329 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440319d0 con 0x7f2c480b3630 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 -- 172.21.15.111:0/3921964329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c480b3630 msgr2=0x7f2c480b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 --2- 172.21.15.111:0/3921964329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c480b3630 0x7f2c480b72a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c4806d5d0 tx=0x7f2c4402a6f0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 -- 172.21.15.111:0/3921964329 shutdown_connections 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 --2- 172.21.15.111:0/3921964329 >> v2:172.21.15.153:3300/0 conn(0x7f2c480b77e0 0x7f2c48161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 --2- 172.21.15.111:0/3921964329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c480b3630 0x7f2c480b72a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 -- 172.21.15.111:0/3921964329 >> 172.21.15.111:0/3921964329 conn(0x7f2c480d42c0 msgr2=0x7f2c480d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 -- 172.21.15.111:0/3921964329 shutdown_connections 2024-05-27T00:40:00.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.021+0000 7f2c50438640 1 -- 172.21.15.111:0/3921964329 wait complete. 2024-05-27T00:40:00.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.022+0000 7f2c50438640 1 Processor -- start 2024-05-27T00:40:00.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.022+0000 7f2c50438640 1 -- start start 2024-05-27T00:40:00.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.022+0000 7f2c50438640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c480b3630 0x7f2c4815f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c480b77e0 0x7f2c480b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 0x7f2c480b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480d34a0 con 0x7f2c48162390 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480d3320 con 0x7f2c480b3630 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2c480d3620 con 0x7f2c480b77e0 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 0x7f2c480b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 0x7f2c480b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46020/0 (socket says 172.21.15.111:46020) 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 -- 172.21.15.111:0/2495880760 learned_addr learned my addr 172.21.15.111:0/2495880760 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e1d4640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c480b3630 0x7f2c4815f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4d9d3640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c480b77e0 0x7f2c480b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 -- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c480b77e0 msgr2=0x7f2c480b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c480b77e0 0x7f2c480b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 -- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c480b3630 msgr2=0x7f2c4815f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c480b3630 0x7f2c4815f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 -- 172.21.15.111:0/2495880760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c480aacb0 con 0x7f2c48162390 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c4e9d5640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 0x7f2c480b2520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b4c0 tx=0x7f2c4000b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40015020 con 0x7f2c48162390 2024-05-27T00:40:00.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c40011850 con 0x7f2c48162390 2024-05-27T00:40:00.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c480aaf90 con 0x7f2c48162390 2024-05-27T00:40:00.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40011c40 con 0x7f2c48162390 2024-05-27T00:40:00.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.023+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c480ab550 con 0x7f2c48162390 2024-05-27T00:40:00.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.024+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c48068680 con 0x7f2c48162390 2024-05-27T00:40:00.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.025+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c40018020 con 0x7f2c48162390 2024-05-27T00:40:00.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.026+0000 7f2c3effd640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c2c06be60 0x7f2c2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.026+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c4008aae0 con 0x7f2c48162390 2024-05-27T00:40:00.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.026+0000 7f2c4e1d4640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c2c06be60 0x7f2c2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.026+0000 7f2c4e1d4640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c2c06be60 0x7f2c2c06e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c44033000 tx=0x7f2c44009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.032+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2c4001f070 con 0x7f2c48162390 2024-05-27T00:40:00.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.212+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c480cff00 con 0x7f2c48162390 2024-05-27T00:40:00.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.213+0000 7f2c3effd640 1 -- 172.21.15.111:0/2495880760 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f2c40059150 con 0x7f2c48162390 2024-05-27T00:40:00.214 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c2c06be60 msgr2=0x7f2c2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c2c06be60 0x7f2c2c06e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c44033000 tx=0x7f2c44009000 comp rx=0 tx=0).stop 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 msgr2=0x7f2c480b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 0x7f2c480b2520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b4c0 tx=0x7f2c4000b980 comp rx=0 tx=0).stop 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 shutdown_connections 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2c2c06be60 0x7f2c2c06e320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c48162390 0x7f2c480b2520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2c480b77e0 0x7f2c480b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 --2- 172.21.15.111:0/2495880760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c480b3630 0x7f2c4815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 >> 172.21.15.111:0/2495880760 conn(0x7f2c480d42c0 msgr2=0x7f2c480b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 shutdown_connections 2024-05-27T00:40:00.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.215+0000 7f2c50438640 1 -- 172.21.15.111:0/2495880760 wait complete. 2024-05-27T00:40:00.227 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:00.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.332+0000 7f2609d70640 1 Processor -- start 2024-05-27T00:40:00.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.332+0000 7f2609d70640 1 -- start start 2024-05-27T00:40:00.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2609d70640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2604161e50 0x7f26041622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2609d70640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040d2c10 0x7f26040d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2609d70640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260406cc70 con 0x7f26040b3630 2024-05-27T00:40:00.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2609d70640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f26040d35e0 con 0x7f2604161e50 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2609d70640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f26040d1e50 con 0x7f26040d2c10 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f25faffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2604161e50 0x7f26041622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f26037fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f26040b3630 0x7f26040b72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59078/0 (socket says 172.21.15.111:59078) 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f26037fe640 1 -- 172.21.15.111:0/2817590660 learned_addr learned my addr 172.21.15.111:0/2817590660 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2603fff640 1 --2- 172.21.15.111:0/2817590660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040d2c10 0x7f26040d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2602ffd640 1 -- 172.21.15.111:0/2817590660 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1693036482 0 0) 0x7f260406cc70 con 0x7f26040b3630 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f2602ffd640 1 -- 172.21.15.111:0/2817590660 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25f0003520 con 0x7f26040b3630 2024-05-27T00:40:00.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f25faffd640 1 -- 172.21.15.111:0/2817590660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040d2c10 msgr2=0x7f26040d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f25faffd640 1 --2- 172.21.15.111:0/2817590660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040d2c10 0x7f26040d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f25faffd640 1 -- 172.21.15.111:0/2817590660 >> v1:172.21.15.111:6789/0 conn(0x7f26040b3630 legacy=0x7f26040b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.333+0000 7f25faffd640 1 -- 172.21.15.111:0/2817590660 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040d2130 con 0x7f2604161e50 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.334+0000 7f25faffd640 1 --2- 172.21.15.111:0/2817590660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f26041622e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0097a0 tx=0x7f25ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b39aafd612d908b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.334+0000 7f2602ffd640 1 -- 172.21.15.111:0/2817590660 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec037070 con 0x7f2604161e50 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.334+0000 7f2602ffd640 1 -- 172.21.15.111:0/2817590660 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec032440 con 0x7f2604161e50 2024-05-27T00:40:00.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.334+0000 7f2602ffd640 1 -- 172.21.15.111:0/2817590660 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec032810 con 0x7f2604161e50 2024-05-27T00:40:00.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 -- 172.21.15.111:0/2817590660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 msgr2=0x7f26041622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 --2- 172.21.15.111:0/2817590660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f26041622e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0097a0 tx=0x7f25ec02a180 comp rx=0 tx=0).stop 2024-05-27T00:40:00.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 -- 172.21.15.111:0/2817590660 shutdown_connections 2024-05-27T00:40:00.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 --2- 172.21.15.111:0/2817590660 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040d2c10 0x7f26040d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 --2- 172.21.15.111:0/2817590660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f26041622e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 -- 172.21.15.111:0/2817590660 >> 172.21.15.111:0/2817590660 conn(0x7f26040d42c0 msgr2=0x7f26040d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 -- 172.21.15.111:0/2817590660 shutdown_connections 2024-05-27T00:40:00.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.335+0000 7f2609d70640 1 -- 172.21.15.111:0/2817590660 wait complete. 2024-05-27T00:40:00.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.336+0000 7f2609d70640 1 Processor -- start 2024-05-27T00:40:00.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.336+0000 7f2609d70640 1 -- start start 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2609d70640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b3630 0x7f26040ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2609d70640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26040d2c10 0x7f260414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2609d70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f260415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2609d70640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26040b79b0 con 0x7f2604161e50 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2609d70640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f26040d26f0 con 0x7f26040d2c10 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2609d70640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f260406bfb0 con 0x7f26040b3630 2024-05-27T00:40:00.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2603fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f260415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2603fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f260415bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46036/0 (socket says 172.21.15.111:46036) 2024-05-27T00:40:00.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.337+0000 7f2603fff640 1 -- 172.21.15.111:0/1947688426 learned_addr learned my addr 172.21.15.111:0/1947688426 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:00.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2603fff640 1 -- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b3630 msgr2=0x7f26040ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2603fff640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b3630 0x7f26040ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2603fff640 1 -- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26040d2c10 msgr2=0x7f260414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2603fff640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26040d2c10 0x7f260414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2603fff640 1 -- 172.21.15.111:0/1947688426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040c9600 con 0x7f2604161e50 2024-05-27T00:40:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2603fff640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f260415bd60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f25fc007e40 tx=0x7f25fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc014620 con 0x7f2604161e50 2024-05-27T00:40:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25fc019070 con 0x7f2604161e50 2024-05-27T00:40:00.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26040c98a0 con 0x7f2604161e50 2024-05-27T00:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc014a00 con 0x7f2604161e50 2024-05-27T00:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.338+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26040c9e60 con 0x7f2604161e50 2024-05-27T00:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.339+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2604068680 con 0x7f2604161e50 2024-05-27T00:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.339+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25fc014ba0 con 0x7f2604161e50 2024-05-27T00:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.340+0000 7f2600ff9640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f25e406be30 0x7f25e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.340+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f25fc08bac0 con 0x7f2604161e50 2024-05-27T00:40:00.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.340+0000 7f26037fe640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f25e406be30 0x7f25e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.341+0000 7f26037fe640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f25e406be30 0x7f25e406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000f60 tx=0x7f25f40078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.344+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f25fc05a130 con 0x7f2604161e50 2024-05-27T00:40:00.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.519+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f26040cef10 con 0x7f2604161e50 2024-05-27T00:40:00.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.519+0000 7f2600ff9640 1 -- 172.21.15.111:0/1947688426 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f25fc059a30 con 0x7f2604161e50 2024-05-27T00:40:00.521 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:00.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f25e406be30 msgr2=0x7f25e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f25e406be30 0x7f25e406e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000f60 tx=0x7f25f40078f0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 msgr2=0x7f260415bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f260415bd60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f25fc007e40 tx=0x7f25fc00e4b0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 shutdown_connections 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f25e406be30 0x7f25e406e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2604161e50 0x7f260415bd60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26040d2c10 0x7f260414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 --2- 172.21.15.111:0/1947688426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f26040b3630 0x7f26040ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.521+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 >> 172.21.15.111:0/1947688426 conn(0x7f26040d42c0 msgr2=0x7f26040b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.522+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 shutdown_connections 2024-05-27T00:40:00.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.522+0000 7f2609d70640 1 -- 172.21.15.111:0/1947688426 wait complete. 2024-05-27T00:40:00.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:00.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.635+0000 7f50dc246640 1 Processor -- start 2024-05-27T00:40:00.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.635+0000 7f50dc246640 1 -- start start 2024-05-27T00:40:00.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50dc246640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f50d406bc20 0x7f50d406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50dc246640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d4071d10 con 0x7f50d4151740 2024-05-27T00:40:00.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50dc246640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d4071fe0 con 0x7f50d40cf030 2024-05-27T00:40:00.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50dc246640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f50d40722c0 con 0x7f50d406bc20 2024-05-27T00:40:00.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50d97e1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f50d406bc20 0x7f50d406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50d9fe2640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f50d4151740 0x7f50d4071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41748/0 (socket says 172.21.15.111:41748) 2024-05-27T00:40:00.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.636+0000 7f50d9fe2640 1 -- 172.21.15.111:0/3622929221 learned_addr learned my addr 172.21.15.111:0/3622929221 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:00.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.637+0000 7f50d97e1640 1 -- 172.21.15.111:0/3622929221 >> v1:172.21.15.153:6789/0 conn(0x7f50d40cf030 legacy=0x7f50d40cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.637+0000 7f50d97e1640 1 -- 172.21.15.111:0/3622929221 >> v1:172.21.15.111:6789/0 conn(0x7f50d4151740 legacy=0x7f50d4071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.639 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.637+0000 7f50d97e1640 1 -- 172.21.15.111:0/3622929221 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40725a0 con 0x7f50d406bc20 2024-05-27T00:40:00.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.637+0000 7f50d97e1640 1 --2- 172.21.15.111:0/3622929221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d406bc20 0x7f50d406c0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097a0 tx=0x7f50c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8246c4fde097ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.640 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50d8fe0640 1 -- 172.21.15.111:0/3622929221 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4037070 con 0x7f50d406bc20 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50d8fe0640 1 -- 172.21.15.111:0/3622929221 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c4032440 con 0x7f50d406bc20 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50d8fe0640 1 -- 172.21.15.111:0/3622929221 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4032810 con 0x7f50d406bc20 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 -- 172.21.15.111:0/3622929221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d406bc20 msgr2=0x7f50d406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 --2- 172.21.15.111:0/3622929221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d406bc20 0x7f50d406c0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097a0 tx=0x7f50c402a230 comp rx=0 tx=0).stop 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 -- 172.21.15.111:0/3622929221 shutdown_connections 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 --2- 172.21.15.111:0/3622929221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d406bc20 0x7f50d406c0a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 -- 172.21.15.111:0/3622929221 >> 172.21.15.111:0/3622929221 conn(0x7f50d40b1c00 msgr2=0x7f50d40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 -- 172.21.15.111:0/3622929221 shutdown_connections 2024-05-27T00:40:00.641 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.638+0000 7f50dc246640 1 -- 172.21.15.111:0/3622929221 wait complete. 2024-05-27T00:40:00.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.639+0000 7f50dc246640 1 Processor -- start 2024-05-27T00:40:00.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.639+0000 7f50dc246640 1 -- start start 2024-05-27T00:40:00.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50dc246640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50d406bc20 0x7f50d41f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50dc246640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d40cf030 0x7f50d40a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50dc246640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 0x7f50d414df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50dc246640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4151b20 con 0x7f50d4151740 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50dc246640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f50d40cfb90 con 0x7f50d40cf030 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50dc246640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f50d415af10 con 0x7f50d406bc20 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50da7e3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 0x7f50d414df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50da7e3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 0x7f50d414df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42006/0 (socket says 172.21.15.111:42006) 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50da7e3640 1 -- 172.21.15.111:0/1465918793 learned_addr learned my addr 172.21.15.111:0/1465918793 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50d97e1640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d40cf030 0x7f50d40a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.640+0000 7f50d9fe2640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50d406bc20 0x7f50d41f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50da7e3640 1 -- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50d406bc20 msgr2=0x7f50d41f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50da7e3640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50d406bc20 0x7f50d41f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50da7e3640 1 -- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d40cf030 msgr2=0x7f50d40a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50da7e3640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d40cf030 0x7f50d40a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50da7e3640 1 -- 172.21.15.111:0/1465918793 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d41f4640 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50da7e3640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 0x7f50d414df80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f50cc001eb0 tx=0x7f50cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc017070 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50cc0052d0 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d41f30c0 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc003810 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.641+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d41f3900 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.642+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d406fb70 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.642+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50cc01a070 con 0x7f50d4151740 2024-05-27T00:40:00.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.643+0000 7f50ca7fc640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50b806be30 0x7f50b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.643+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f50cc08a4d0 con 0x7f50d4151740 2024-05-27T00:40:00.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.643+0000 7f50d9fe2640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50b806be30 0x7f50b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.643+0000 7f50d9fe2640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50b806be30 0x7f50b806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005c00 tx=0x7f50d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.647+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50d406fb70 con 0x7f50d4151740 2024-05-27T00:40:00.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.821+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50d4068680 con 0x7f50d4151740 2024-05-27T00:40:00.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.822+0000 7f50ca7fc640 1 -- 172.21.15.111:0/1465918793 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f50d41f30c0 con 0x7f50d4151740 2024-05-27T00:40:00.823 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:00.824 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:39:27.597841+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:00.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50b806be30 msgr2=0x7f50b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50b806be30 0x7f50b806e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005c00 tx=0x7f50d0009040 comp rx=0 tx=0).stop 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 msgr2=0x7f50d414df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 0x7f50d414df80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f50cc001eb0 tx=0x7f50cc00c890 comp rx=0 tx=0).stop 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 shutdown_connections 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50b806be30 0x7f50b806e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50d4151740 0x7f50d414df80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50d40cf030 0x7f50d40a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 --2- 172.21.15.111:0/1465918793 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f50d406bc20 0x7f50d41f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 >> 172.21.15.111:0/1465918793 conn(0x7f50d40b1c00 msgr2=0x7f50d4158e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.824+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 shutdown_connections 2024-05-27T00:40:00.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.825+0000 7f50dc246640 1 -- 172.21.15.111:0/1465918793 wait complete. 2024-05-27T00:40:00.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:00.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:00.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.939+0000 7fea04f22640 1 Processor -- start 2024-05-27T00:40:00.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.940+0000 7fea04f22640 1 -- start start 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.940+0000 7fea04f22640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea0006b390 0x7fea000cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fea04f22640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea000b36e0 con 0x7fea000cdc80 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fea04f22640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea000b39c0 con 0x7fea00144000 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fea04f22640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fea000b42f0 con 0x7fea0006b390 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fed9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fea000cdc80 0x7fea000726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41754/0 (socket says 172.21.15.111:41754) 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fed9d640 1 -- 172.21.15.111:0/2897054711 learned_addr learned my addr 172.21.15.111:0/2897054711 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fdd9b640 1 --2- 172.21.15.111:0/2897054711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea0006b390 0x7fea000cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fd59a640 1 -- 172.21.15.111:0/2897054711 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937285200 0 0) 0x7fea000b39c0 con 0x7fea00144000 2024-05-27T00:40:00.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fd59a640 1 -- 172.21.15.111:0/2897054711 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9ec003520 con 0x7fea00144000 2024-05-27T00:40:00.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fdd9b640 1 -- 172.21.15.111:0/2897054711 >> v1:172.21.15.153:6789/0 conn(0x7fea00144000 legacy=0x7fea000b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fdd9b640 1 -- 172.21.15.111:0/2897054711 >> v1:172.21.15.111:6789/0 conn(0x7fea000cdc80 legacy=0x7fea000726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.941+0000 7fe9fdd9b640 1 -- 172.21.15.111:0/2897054711 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea000b3ca0 con 0x7fea0006b390 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.942+0000 7fe9fdd9b640 1 --2- 172.21.15.111:0/2897054711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea0006b390 0x7fea000cd740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80097d0 tx=0x7fe9e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb8ed3c0cde616f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.942+0000 7fe9fd59a640 1 -- 172.21.15.111:0/2897054711 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8037070 con 0x7fea0006b390 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.942+0000 7fe9fd59a640 1 -- 172.21.15.111:0/2897054711 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e8032440 con 0x7fea0006b390 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.942+0000 7fe9fd59a640 1 -- 172.21.15.111:0/2897054711 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8032810 con 0x7fea0006b390 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 -- 172.21.15.111:0/2897054711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea0006b390 msgr2=0x7fea000cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 --2- 172.21.15.111:0/2897054711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea0006b390 0x7fea000cd740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80097d0 tx=0x7fe9e802a210 comp rx=0 tx=0).stop 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 -- 172.21.15.111:0/2897054711 shutdown_connections 2024-05-27T00:40:00.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 --2- 172.21.15.111:0/2897054711 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea0006b390 0x7fea000cd740 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 -- 172.21.15.111:0/2897054711 >> 172.21.15.111:0/2897054711 conn(0x7fea000b1c00 msgr2=0x7fea000b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:00.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 -- 172.21.15.111:0/2897054711 shutdown_connections 2024-05-27T00:40:00.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.943+0000 7fea04f22640 1 -- 172.21.15.111:0/2897054711 wait complete. 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.944+0000 7fea04f22640 1 Processor -- start 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.944+0000 7fea04f22640 1 -- start start 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fea04f22640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 0x7fea001f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fea04f22640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea000cdc80 0x7fea000aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fea04f22640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea00144000 0x7fea0006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fea04f22640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea00072f60 con 0x7fea0006b390 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fea04f22640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fea00072de0 con 0x7fea000cdc80 2024-05-27T00:40:00.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fea04f22640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fea000b4780 con 0x7fea00144000 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 0x7fea001f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 0x7fea001f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42012/0 (socket says 172.21.15.111:42012) 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 -- 172.21.15.111:0/2217398454 learned_addr learned my addr 172.21.15.111:0/2217398454 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fed9d640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea00144000 0x7fea0006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fdd9b640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea000cdc80 0x7fea000aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 -- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea00144000 msgr2=0x7fea0006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea00144000 0x7fea0006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 -- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea000cdc80 msgr2=0x7fea000aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea000cdc80 0x7fea000aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 -- 172.21.15.111:0/2217398454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea000af030 con 0x7fea0006b390 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fdd9b640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea000cdc80 0x7fea000aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:00.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9fe59c640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 0x7fea001f4590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000c430 tx=0x7fe9f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.945+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0017070 con 0x7fea0006b390 2024-05-27T00:40:00.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.946+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f00052f0 con 0x7fea0006b390 2024-05-27T00:40:00.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.946+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea001f7270 con 0x7fea0006b390 2024-05-27T00:40:00.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.946+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003830 con 0x7fea0006b390 2024-05-27T00:40:00.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.946+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea001f7830 con 0x7fea0006b390 2024-05-27T00:40:00.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.947+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea00068680 con 0x7fea0006b390 2024-05-27T00:40:00.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.947+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9f001a070 con 0x7fea0006b390 2024-05-27T00:40:00.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.947+0000 7fe9f6ffd640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe9e006be30 0x7fe9e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:00.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.947+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe9f008a4a0 con 0x7fea0006b390 2024-05-27T00:40:00.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.948+0000 7fe9fdd9b640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe9e006be30 0x7fe9e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:00.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.948+0000 7fe9fdd9b640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe9e006be30 0x7fe9e006e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8034000 tx=0x7fe9e802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:00.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:00.952+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fea000af030 con 0x7fea0006b390 2024-05-27T00:40:01.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.133+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea00074cf0 con 0x7fea0006b390 2024-05-27T00:40:01.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.133+0000 7fe9f6ffd640 1 -- 172.21.15.111:0/2217398454 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe9f0058550 con 0x7fea0006b390 2024-05-27T00:40:01.134 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:01.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe9e006be30 msgr2=0x7fe9e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe9e006be30 0x7fe9e006e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8034000 tx=0x7fe9e802e000 comp rx=0 tx=0).stop 2024-05-27T00:40:01.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 msgr2=0x7fea001f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 0x7fea001f4590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000c430 tx=0x7fe9f000c8f0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 shutdown_connections 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fe9e006be30 0x7fe9e006e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fea00144000 0x7fea0006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fea000cdc80 0x7fea000aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 --2- 172.21.15.111:0/2217398454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0006b390 0x7fea001f4590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 >> 172.21.15.111:0/2217398454 conn(0x7fea000b1c00 msgr2=0x7fea000b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 shutdown_connections 2024-05-27T00:40:01.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.135+0000 7fea04f22640 1 -- 172.21.15.111:0/2217398454 wait complete. 2024-05-27T00:40:01.147 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:01.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.248+0000 7f343cbd5640 1 Processor -- start 2024-05-27T00:40:01.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.248+0000 7f343cbd5640 1 -- start start 2024-05-27T00:40:01.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f343cbd5640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3438144000 0x7f34380b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f343cbd5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f34380cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f343cbd5640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34380b3440 con 0x7f34380cdc80 2024-05-27T00:40:01.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f343cbd5640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f34380b42f0 con 0x7f3438144000 2024-05-27T00:40:01.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f343cbd5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f34380b36e0 con 0x7f343806b390 2024-05-27T00:40:01.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f3435d9b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f34380cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f343659c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3438144000 0x7f34380b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f3435d9b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f34380cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:56824/0 (socket says 172.21.15.111:56824) 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.249+0000 7f3435d9b640 1 -- 172.21.15.111:0/2810059068 learned_addr learned my addr 172.21.15.111:0/2810059068 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.250+0000 7f3435d9b640 1 -- 172.21.15.111:0/2810059068 >> v2:172.21.15.153:3300/0 conn(0x7f3438144000 msgr2=0x7f34380b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.250+0000 7f3435d9b640 1 --2- 172.21.15.111:0/2810059068 >> v2:172.21.15.153:3300/0 conn(0x7f3438144000 0x7f34380b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.250+0000 7f3435d9b640 1 -- 172.21.15.111:0/2810059068 >> v1:172.21.15.153:6789/0 conn(0x7f34380cdc80 legacy=0x7f34380726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.250+0000 7f3435d9b640 1 -- 172.21.15.111:0/2810059068 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34380b39c0 con 0x7f343806b390 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.250+0000 7f343659c640 1 --2- 172.21.15.111:0/2810059068 >> v2:172.21.15.153:3300/0 conn(0x7f3438144000 0x7f34380b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.250+0000 7f3435d9b640 1 --2- 172.21.15.111:0/2810059068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f34380cd740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3420009870 tx=0x7f342002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dbaa783bbaf345 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343559a640 1 -- 172.21.15.111:0/2810059068 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420037070 con 0x7f343806b390 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343559a640 1 -- 172.21.15.111:0/2810059068 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3420032440 con 0x7f343806b390 2024-05-27T00:40:01.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343559a640 1 -- 172.21.15.111:0/2810059068 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420032810 con 0x7f343806b390 2024-05-27T00:40:01.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 -- 172.21.15.111:0/2810059068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 msgr2=0x7f34380cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 --2- 172.21.15.111:0/2810059068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f34380cd740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3420009870 tx=0x7f342002a270 comp rx=0 tx=0).stop 2024-05-27T00:40:01.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 -- 172.21.15.111:0/2810059068 shutdown_connections 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 --2- 172.21.15.111:0/2810059068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f34380cd740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 --2- 172.21.15.111:0/2810059068 >> v2:172.21.15.153:3300/0 conn(0x7f3438144000 0x7f34380b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 -- 172.21.15.111:0/2810059068 >> 172.21.15.111:0/2810059068 conn(0x7f34380b1c00 msgr2=0x7f34380b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 -- 172.21.15.111:0/2810059068 shutdown_connections 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.251+0000 7f343cbd5640 1 -- 172.21.15.111:0/2810059068 wait complete. 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 Processor -- start 2024-05-27T00:40:01.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 -- start start 2024-05-27T00:40:01.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f343813c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34380cdc80 0x7f34381e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 0x7f3438142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3438072de0 con 0x7f3438144000 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34380b3f80 con 0x7f34380cdc80 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f343cbd5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3438072f60 con 0x7f343806b390 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.253+0000 7f3436d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 0x7f3438142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 0x7f3438142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42028/0 (socket says 172.21.15.111:42028) 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 -- 172.21.15.111:0/1996263982 learned_addr learned my addr 172.21.15.111:0/1996263982 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3435d9b640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34380cdc80 0x7f34381e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f343659c640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f343813c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 -- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 msgr2=0x7f343813c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f343813c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 -- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34380cdc80 msgr2=0x7f34381e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34380cdc80 0x7f34381e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 -- 172.21.15.111:0/1996263982 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34381e3630 con 0x7f3438144000 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3435d9b640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34380cdc80 0x7f34381e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f343659c640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f343813c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f3436d9d640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 0x7f3438142390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f343000c3c0 tx=0x7f343000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430017070 con 0x7f3438144000 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34300052d0 con 0x7f3438144000 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34381e3910 con 0x7f3438144000 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430003830 con 0x7f3438144000 2024-05-27T00:40:01.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.254+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34381e3ed0 con 0x7f3438144000 2024-05-27T00:40:01.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.255+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3438068680 con 0x7f3438144000 2024-05-27T00:40:01.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.256+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f343001a070 con 0x7f3438144000 2024-05-27T00:40:01.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.256+0000 7f342effd640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f341806be60 0x7f341806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.256+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f343008a530 con 0x7f3438144000 2024-05-27T00:40:01.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.256+0000 7f343659c640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f341806be60 0x7f341806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.257+0000 7f343659c640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f341806be60 0x7f341806e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f342800bfd0 tx=0x7f342800d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.260+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3430058ac0 con 0x7f3438144000 2024-05-27T00:40:01.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.434+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f343813e620 con 0x7f3438144000 2024-05-27T00:40:01.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.435+0000 7f342effd640 1 -- 172.21.15.111:0/1996263982 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f34300583c0 con 0x7f3438144000 2024-05-27T00:40:01.436 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:01.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f341806be60 msgr2=0x7f341806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f341806be60 0x7f341806e320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f342800bfd0 tx=0x7f342800d040 comp rx=0 tx=0).stop 2024-05-27T00:40:01.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 msgr2=0x7f3438142390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 0x7f3438142390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f343000c3c0 tx=0x7f343000c880 comp rx=0 tx=0).stop 2024-05-27T00:40:01.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 shutdown_connections 2024-05-27T00:40:01.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f341806be60 0x7f341806e320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3438144000 0x7f3438142390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34380cdc80 0x7f34381e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 --2- 172.21.15.111:0/1996263982 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f343806b390 0x7f343813c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 >> 172.21.15.111:0/1996263982 conn(0x7f34380b1c00 msgr2=0x7f34380af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:01.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 shutdown_connections 2024-05-27T00:40:01.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.437+0000 7f343cbd5640 1 -- 172.21.15.111:0/1996263982 wait complete. 2024-05-27T00:40:01.449 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:01.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.550+0000 7fcd021d5640 1 Processor -- start 2024-05-27T00:40:01.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.550+0000 7fcd021d5640 1 -- start start 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fcd021d5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fccfc151740 0x7fccfc071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fcd021d5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc071b40 0x7fccfc06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fcd021d5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccfc071fe0 con 0x7fccfc06c160 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fcd021d5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc06c5d0 con 0x7fccfc151740 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fcd021d5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc0722c0 con 0x7fccfc071b40 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfb7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fccfc151740 0x7fccfc071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfbfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fccfc06c160 0x7fccfc0cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41760/0 (socket says 172.21.15.111:41760) 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfbfff640 1 -- 172.21.15.111:0/3129998761 learned_addr learned my addr 172.21.15.111:0/3129998761 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfb7fe640 1 -- 172.21.15.111:0/3129998761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc071b40 msgr2=0x7fccfc06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfb7fe640 1 --2- 172.21.15.111:0/3129998761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc071b40 0x7fccfc06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfb7fe640 1 -- 172.21.15.111:0/3129998761 >> v1:172.21.15.111:6789/0 conn(0x7fccfc06c160 legacy=0x7fccfc0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfaffd640 1 -- 172.21.15.111:0/3129998761 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661693425 0 0) 0x7fccfc071fe0 con 0x7fccfc06c160 2024-05-27T00:40:01.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.551+0000 7fccfb7fe640 1 -- 172.21.15.111:0/3129998761 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0725a0 con 0x7fccfc151740 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.552+0000 7fccfb7fe640 1 --2- 172.21.15.111:0/3129998761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc151740 0x7fccfc071600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fccec0097a0 tx=0x7fccec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a210609d3c6ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.552+0000 7fccfaffd640 1 -- 172.21.15.111:0/3129998761 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec037070 con 0x7fccfc151740 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.552+0000 7fccfaffd640 1 -- 172.21.15.111:0/3129998761 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec032440 con 0x7fccfc151740 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.552+0000 7fccfaffd640 1 -- 172.21.15.111:0/3129998761 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec032810 con 0x7fccfc151740 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 -- 172.21.15.111:0/3129998761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc151740 msgr2=0x7fccfc071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 --2- 172.21.15.111:0/3129998761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc151740 0x7fccfc071600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fccec0097a0 tx=0x7fccec02a180 comp rx=0 tx=0).stop 2024-05-27T00:40:01.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 -- 172.21.15.111:0/3129998761 shutdown_connections 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 --2- 172.21.15.111:0/3129998761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc071b40 0x7fccfc06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 --2- 172.21.15.111:0/3129998761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc151740 0x7fccfc071600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 -- 172.21.15.111:0/3129998761 >> 172.21.15.111:0/3129998761 conn(0x7fccfc0b1c00 msgr2=0x7fccfc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 -- 172.21.15.111:0/3129998761 shutdown_connections 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.553+0000 7fcd021d5640 1 -- 172.21.15.111:0/3129998761 wait complete. 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.554+0000 7fcd021d5640 1 Processor -- start 2024-05-27T00:40:01.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.554+0000 7fcd021d5640 1 -- start start 2024-05-27T00:40:01.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc06c160 0x7fccfc145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 0x7fccfc14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc151740 0x7fccfc14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc15af10 con 0x7fccfc071b40 2024-05-27T00:40:01.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc151b20 con 0x7fccfc151740 2024-05-27T00:40:01.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc15b090 con 0x7fccfc06c160 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 0x7fccfc14a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 0x7fccfc14a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42046/0 (socket says 172.21.15.111:42046) 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 -- 172.21.15.111:0/18562508 learned_addr learned my addr 172.21.15.111:0/18562508 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccfbfff640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc151740 0x7fccfc14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 -- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc06c160 msgr2=0x7fccfc145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc06c160 0x7fccfc145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 -- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc151740 msgr2=0x7fccfc14ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc151740 0x7fccfc14ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 -- 172.21.15.111:0/18562508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc14b4b0 con 0x7fccfc071b40 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf3fff640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 0x7fccfc14a640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcce400b4a0 tx=0x7fcce400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4014070 con 0x7fccfc071b40 2024-05-27T00:40:01.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce4010610 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc147ae0 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4010a00 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.555+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc14ede0 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.556+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc068680 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.557+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcce4017070 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.557+0000 7fccf8ff9640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fccdc06be30 0x7fccdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.557+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fcce408ac10 con 0x7fccfc071b40 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.558+0000 7fccfb7fe640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fccdc06be30 0x7fccdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.558+0000 7fccfb7fe640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fccdc06be30 0x7fccdc06e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fccec034000 tx=0x7fccec02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.561+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcce4059230 con 0x7fccfc071b40 2024-05-27T00:40:01.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.741+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccfc0d0490 con 0x7fccfc071b40 2024-05-27T00:40:01.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.741+0000 7fccf8ff9640 1 -- 172.21.15.111:0/18562508 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fcce4058b30 con 0x7fccfc071b40 2024-05-27T00:40:01.743 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.743+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fccdc06be30 msgr2=0x7fccdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.743+0000 7fcd021d5640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fccdc06be30 0x7fccdc06e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fccec034000 tx=0x7fccec02c000 comp rx=0 tx=0).stop 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.743+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 msgr2=0x7fccfc14a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.743+0000 7fcd021d5640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 0x7fccfc14a640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcce400b4a0 tx=0x7fcce400b960 comp rx=0 tx=0).stop 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 shutdown_connections 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fccdc06be30 0x7fccdc06e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fccfc151740 0x7fccfc14ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccfc071b40 0x7fccfc14a640 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 --2- 172.21.15.111:0/18562508 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fccfc06c160 0x7fccfc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 >> 172.21.15.111:0/18562508 conn(0x7fccfc0b1c00 msgr2=0x7fccfc1591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:01.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 shutdown_connections 2024-05-27T00:40:01.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.744+0000 7fcd021d5640 1 -- 172.21.15.111:0/18562508 wait complete. 2024-05-27T00:40:01.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:01.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.857+0000 7f5b60909640 1 Processor -- start 2024-05-27T00:40:01.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.857+0000 7f5b60909640 1 -- start start 2024-05-27T00:40:01.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b60909640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5b5c13ef60 0x7f5b5c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b60909640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b5c0ccfd0 0x7f5b5c141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b60909640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5c142650 con 0x7f5b5c13f340 2024-05-27T00:40:01.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b60909640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c0cf210 con 0x7f5b5c0ccfd0 2024-05-27T00:40:01.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b60909640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c0cebc0 con 0x7f5b5c13ef60 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b5c0ccfd0 0x7f5b5c141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5b5c0ccfd0 0x7f5b5c141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42054/0 (socket says 172.21.15.111:42054) 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/106352679 learned_addr learned my addr 172.21.15.111:0/106352679 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/106352679 >> v2:172.21.15.153:3300/0 conn(0x7f5b5c13ef60 msgr2=0x7f5b5c0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5a59c640 1 --2- 172.21.15.111:0/106352679 >> v2:172.21.15.153:3300/0 conn(0x7f5b5c13ef60 0x7f5b5c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 --2- 172.21.15.111:0/106352679 >> v2:172.21.15.153:3300/0 conn(0x7f5b5c13ef60 0x7f5b5c0cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/106352679 >> v1:172.21.15.153:6789/0 conn(0x7f5b5c13f340 legacy=0x7f5b5c0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.858+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/106352679 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c0ceea0 con 0x7f5b5c0ccfd0 2024-05-27T00:40:01.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.859+0000 7f5b5ad9d640 1 --2- 172.21.15.111:0/106352679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c141920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0742a0 tx=0x7f5b4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2ce5e0aaa11d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.859+0000 7f5b5959a640 1 -- 172.21.15.111:0/106352679 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c036070 con 0x7f5b5c0ccfd0 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.859+0000 7f5b5959a640 1 -- 172.21.15.111:0/106352679 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c031600 con 0x7f5b5c0ccfd0 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.859+0000 7f5b5959a640 1 -- 172.21.15.111:0/106352679 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0319d0 con 0x7f5b5c0ccfd0 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 -- 172.21.15.111:0/106352679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c0ccfd0 msgr2=0x7f5b5c141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 --2- 172.21.15.111:0/106352679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c141920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0742a0 tx=0x7f5b4c02a6f0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 -- 172.21.15.111:0/106352679 shutdown_connections 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 --2- 172.21.15.111:0/106352679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c141920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 --2- 172.21.15.111:0/106352679 >> v2:172.21.15.153:3300/0 conn(0x7f5b5c13ef60 0x7f5b5c0cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 -- 172.21.15.111:0/106352679 >> 172.21.15.111:0/106352679 conn(0x7f5b5c0b1c00 msgr2=0x7f5b5c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 -- 172.21.15.111:0/106352679 shutdown_connections 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.860+0000 7f5b60909640 1 -- 172.21.15.111:0/106352679 wait complete. 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.861+0000 7f5b60909640 1 Processor -- start 2024-05-27T00:40:01.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.861+0000 7f5b60909640 1 -- start start 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b60909640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b60909640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b5c13ef60 0x7f5b5c13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b60909640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 0x7f5b5c1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b60909640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c141e60 con 0x7f5b5c13f340 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b60909640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0ce890 con 0x7f5b5c0ccfd0 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b60909640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c141fe0 con 0x7f5b5c13ef60 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 0x7f5b5c1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 0x7f5b5c1f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42062/0 (socket says 172.21.15.111:42062) 2024-05-27T00:40:01.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/1034339903 learned_addr learned my addr 172.21.15.111:0/1034339903 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5a59c640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b59d9b640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b5c13ef60 0x7f5b5c13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b5c13ef60 msgr2=0x7f5b5c13e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b5c13ef60 0x7f5b5c13e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b5c0ccfd0 msgr2=0x7f5b5c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.862+0000 7f5b5ad9d640 1 -- 172.21.15.111:0/1034339903 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c15b600 con 0x7f5b5c13f340 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.863+0000 7f5b5ad9d640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 0x7f5b5c1f1c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009f40 tx=0x7f5b4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.863+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c036050 con 0x7f5b5c13f340 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.863+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c042070 con 0x7f5b5c13f340 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.863+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c15a830 con 0x7f5b5c13f340 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.863+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c009510 con 0x7f5b5c13f340 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.863+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c15adf0 con 0x7f5b5c13f340 2024-05-27T00:40:01.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.864+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5c068680 con 0x7f5b5c13f340 2024-05-27T00:40:01.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.864+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b4c007bf0 con 0x7f5b5c13f340 2024-05-27T00:40:01.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.865+0000 7f5b52ffd640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b3c06be30 0x7f5b3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:01.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.865+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f5b4c0ac190 con 0x7f5b5c13f340 2024-05-27T00:40:01.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.865+0000 7f5b5a59c640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b3c06be30 0x7f5b3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:01.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.866+0000 7f5b5a59c640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b3c06be30 0x7f5b3c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400aac0 tx=0x7f5b54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:01.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:01.869+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5b4c07a1f0 con 0x7f5b5c13f340 2024-05-27T00:40:02.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.045+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5b5c0b23b0 con 0x7f5b5c13f340 2024-05-27T00:40:02.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.046+0000 7f5b52ffd640 1 -- 172.21.15.111:0/1034339903 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f5b4c07e4f0 con 0x7f5b5c13f340 2024-05-27T00:40:02.047 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b3c06be30 msgr2=0x7f5b3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b3c06be30 0x7f5b3c06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400aac0 tx=0x7f5b54008040 comp rx=0 tx=0).stop 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 msgr2=0x7f5b5c1f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 0x7f5b5c1f1c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009f40 tx=0x7f5b4c004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 shutdown_connections 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5b3c06be30 0x7f5b3c06e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b5c13f340 0x7f5b5c1f1c40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5b5c13ef60 0x7f5b5c13e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 --2- 172.21.15.111:0/1034339903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b5c0ccfd0 0x7f5b5c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 >> 172.21.15.111:0/1034339903 conn(0x7f5b5c0b1c00 msgr2=0x7f5b5c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 shutdown_connections 2024-05-27T00:40:02.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.048+0000 7f5b60909640 1 -- 172.21.15.111:0/1034339903 wait complete. 2024-05-27T00:40:02.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:02.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.158+0000 7f284bc10640 1 Processor -- start 2024-05-27T00:40:02.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.158+0000 7f284bc10640 1 -- start start 2024-05-27T00:40:02.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284bc10640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2844144000 0x7f28440b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284bc10640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28440726d0 0x7f2844072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284bc10640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28440b3440 con 0x7f284406b390 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284bc10640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f2844073080 con 0x7f28440726d0 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284bc10640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f28440b42f0 con 0x7f2844144000 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284a1ad640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28440726d0 0x7f2844072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f28491ab640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f284406b390 0x7f28440cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41766/0 (socket says 172.21.15.111:41766) 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284a1ad640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f28440726d0 0x7f2844072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42064/0 (socket says 172.21.15.111:42064) 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f28491ab640 1 -- 172.21.15.111:0/3881214944 learned_addr learned my addr 172.21.15.111:0/3881214944 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f28499ac640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2844144000 0x7f28440b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284a1ad640 1 -- 172.21.15.111:0/3881214944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2844144000 msgr2=0x7f28440b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284a1ad640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2844144000 0x7f28440b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284a1ad640 1 -- 172.21.15.111:0/3881214944 >> v1:172.21.15.111:6789/0 conn(0x7f284406b390 legacy=0x7f28440cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f284a1ad640 1 -- 172.21.15.111:0/3881214944 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28440b36e0 con 0x7f28440726d0 2024-05-27T00:40:02.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f28489aa640 1 -- 172.21.15.111:0/3881214944 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922976580 0 0) 0x7f28440b3440 con 0x7f284406b390 2024-05-27T00:40:02.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.159+0000 7f28499ac640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2844144000 0x7f28440b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:02.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.160+0000 7f284a1ad640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440726d0 0x7f2844072b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f283c009d90 tx=0x7f283c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad8c956f1b58b97 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.160+0000 7f28489aa640 1 -- 172.21.15.111:0/3881214944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c007740 con 0x7f28440726d0 2024-05-27T00:40:02.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.160+0000 7f28489aa640 1 -- 172.21.15.111:0/3881214944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c0078e0 con 0x7f28440726d0 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.160+0000 7f28489aa640 1 -- 172.21.15.111:0/3881214944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c007cb0 con 0x7f28440726d0 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 -- 172.21.15.111:0/3881214944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440726d0 msgr2=0x7f2844072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440726d0 0x7f2844072b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f283c009d90 tx=0x7f283c02a660 comp rx=0 tx=0).stop 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 -- 172.21.15.111:0/3881214944 shutdown_connections 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f28440726d0 0x7f2844072b40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 --2- 172.21.15.111:0/3881214944 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2844144000 0x7f28440b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 -- 172.21.15.111:0/3881214944 >> 172.21.15.111:0/3881214944 conn(0x7f28440b1c00 msgr2=0x7f28440b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 -- 172.21.15.111:0/3881214944 shutdown_connections 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.161+0000 7f284bc10640 1 -- 172.21.15.111:0/3881214944 wait complete. 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.162+0000 7f284bc10640 1 Processor -- start 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.162+0000 7f284bc10640 1 -- start start 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284406b390 0x7f28441e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28440726d0 0x7f28441e1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 0x7f28440cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f28440b3f80 con 0x7f2844144000 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f28440b3ca0 con 0x7f284406b390 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f28440cde50 con 0x7f28440726d0 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 0x7f28440cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 0x7f28440cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42080/0 (socket says 172.21.15.111:42080) 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 -- 172.21.15.111:0/5329805 learned_addr learned my addr 172.21.15.111:0/5329805 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 -- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28440726d0 msgr2=0x7f28441e1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f28491ab640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28440726d0 0x7f28441e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f28499ac640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284406b390 0x7f28441e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28440726d0 0x7f28441e1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 -- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284406b390 msgr2=0x7f28441e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284406b390 0x7f28441e8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 -- 172.21.15.111:0/5329805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28441e1a90 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284a1ad640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 0x7f28440cf6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f283c009d90 tx=0x7f283c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c040070 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c031c40 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c003990 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28441e1d70 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.163+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2844159fe0 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.164+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2844068680 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.165+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f283c039040 con 0x7f2844144000 2024-05-27T00:40:02.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.165+0000 7f283a7fc640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f282806be30 0x7f282806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.165+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f283c0ac4c0 con 0x7f2844144000 2024-05-27T00:40:02.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.165+0000 7f28499ac640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f282806be30 0x7f282806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.166+0000 7f28499ac640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f282806be30 0x7f282806e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f284000b840 tx=0x7f2840008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.169+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f283c07aae0 con 0x7f2844144000 2024-05-27T00:40:02.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.346+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28441e1f40 con 0x7f2844144000 2024-05-27T00:40:02.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.347+0000 7f283a7fc640 1 -- 172.21.15.111:0/5329805 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f283c07a3e0 con 0x7f2844144000 2024-05-27T00:40:02.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:02.349 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:02.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f282806be30 msgr2=0x7f282806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f282806be30 0x7f282806e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f284000b840 tx=0x7f2840008040 comp rx=0 tx=0).stop 2024-05-27T00:40:02.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 msgr2=0x7f28440cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 0x7f28440cf6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f283c009d90 tx=0x7f283c004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:02.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 shutdown_connections 2024-05-27T00:40:02.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f282806be30 0x7f282806e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844144000 0x7f28440cf6c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f28440726d0 0x7f28441e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 --2- 172.21.15.111:0/5329805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284406b390 0x7f28441e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 >> 172.21.15.111:0/5329805 conn(0x7f28440b1c00 msgr2=0x7f28440ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 shutdown_connections 2024-05-27T00:40:02.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.350+0000 7f284bc10640 1 -- 172.21.15.111:0/5329805 wait complete. 2024-05-27T00:40:02.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:02.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:02.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.465+0000 7f58ce3a2640 1 Processor -- start 2024-05-27T00:40:02.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.465+0000 7f58ce3a2640 1 -- start start 2024-05-27T00:40:02.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58ce3a2640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f58c80b3630 0x7f58c80b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58ce3a2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 0x7f58c8161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58ce3a2640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c806cc70 con 0x7f58c8162390 2024-05-27T00:40:02.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58ce3a2640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f58c80d1e50 con 0x7f58c80b3630 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58ce3a2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f58c80d2130 con 0x7f58c80b77e0 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58cc93f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f58c8162390 0x7f58c80d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41770/0 (socket says 172.21.15.111:41770) 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.466+0000 7f58cc93f640 1 -- 172.21.15.111:0/2451149761 learned_addr learned my addr 172.21.15.111:0/2451149761 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787076960 0 0) 0x7f58c806cc70 con 0x7f58c8162390 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58b4003520 con 0x7f58c8162390 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58bffff640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 0x7f58c8161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c7fff640 1 --2- 172.21.15.111:0/2451149761 >> v2:172.21.15.153:3300/0 conn(0x7f58c80b3630 0x7f58c80b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1634694429 0 0) 0x7f58b4003520 con 0x7f58c8162390 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 msgr2=0x7f58c8161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 0x7f58c8161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 >> v2:172.21.15.153:3300/0 conn(0x7f58c80b3630 msgr2=0x7f58c80b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:02.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 --2- 172.21.15.111:0/2451149761 >> v2:172.21.15.153:3300/0 conn(0x7f58c80b3630 0x7f58c80b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80d2410 con 0x7f58c8162390 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c7fff640 1 --2- 172.21.15.111:0/2451149761 >> v2:172.21.15.153:3300/0 conn(0x7f58c80b3630 0x7f58c80b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f58c00019f0 con 0x7f58c8162390 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 >> v1:172.21.15.111:6789/0 conn(0x7f58c8162390 legacy=0x7f58c80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b40039b0 0x7f58b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 0x7f58b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b4004d40 0x7f58b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80d1e50 con 0x7f58b4004d40 2024-05-27T00:40:02.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80d2130 con 0x7f58b40039b0 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f58b400b9c0 con 0x7f58c80b77e0 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58cc93f640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b4004d40 0x7f58b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58bffff640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 0x7f58b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58c7fff640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b40039b0 0x7f58b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58cc93f640 1 -- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 msgr2=0x7f58b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58cc93f640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c80b77e0 0x7f58b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58cc93f640 1 -- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b40039b0 msgr2=0x7f58b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.467+0000 7f58cc93f640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b40039b0 0x7f58b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.468+0000 7f58cc93f640 1 -- 172.21.15.111:0/2451149761 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b4003520 con 0x7f58b4004d40 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.468+0000 7f58cc93f640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b4004d40 0x7f58b40051c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58c000a890 tx=0x7f58b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7127dbea1e8d3a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.468+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c000a4e0 con 0x7f58b4004d40 2024-05-27T00:40:02.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.468+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c002f070 con 0x7f58b4004d40 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.468+0000 7f58c77fe640 1 -- 172.21.15.111:0/2451149761 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c002bcf0 con 0x7f58b4004d40 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58ce3a2640 1 -- 172.21.15.111:0/2451149761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b4004d40 msgr2=0x7f58b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/2451149761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b4004d40 0x7f58b40051c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58c000a890 tx=0x7f58b4005830 comp rx=0 tx=0).stop 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58cc93f640 1 -- 172.21.15.111:0/2451149761 reap_dead start 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58ce3a2640 1 -- 172.21.15.111:0/2451149761 shutdown_connections 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58ce3a2640 1 -- 172.21.15.111:0/2451149761 >> 172.21.15.111:0/2451149761 conn(0x7f58c80d42c0 msgr2=0x7f58c80d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58ce3a2640 1 -- 172.21.15.111:0/2451149761 shutdown_connections 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.469+0000 7f58ce3a2640 1 -- 172.21.15.111:0/2451149761 wait complete. 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.470+0000 7f58ce3a2640 1 Processor -- start 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.470+0000 7f58ce3a2640 1 -- start start 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4004d40 0x7f58c81590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 0x7f58c8160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c815be10 0x7f58c815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80d34a0 con 0x7f58c81607d0 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80d3320 con 0x7f58b4004d40 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f58c80d3620 con 0x7f58c815be10 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 0x7f58c8160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 0x7f58c8160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42096/0 (socket says 172.21.15.111:42096) 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 -- 172.21.15.111:0/1282101753 learned_addr learned my addr 172.21.15.111:0/1282101753 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 -- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c815be10 msgr2=0x7f58c815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c815be10 0x7f58c815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 -- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4004d40 msgr2=0x7f58c81590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4004d40 0x7f58c81590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 -- 172.21.15.111:0/1282101753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80cc450 con 0x7f58c81607d0 2024-05-27T00:40:02.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58bffff640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 0x7f58c8160c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58b0011920 tx=0x7f58b0011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000faf0 con 0x7f58c81607d0 2024-05-27T00:40:02.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b000fc90 con 0x7f58c81607d0 2024-05-27T00:40:02.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c80cc730 con 0x7f58c81607d0 2024-05-27T00:40:02.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.471+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0014440 con 0x7f58c81607d0 2024-05-27T00:40:02.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.472+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c80cccf0 con 0x7f58c81607d0 2024-05-27T00:40:02.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.473+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c8068680 con 0x7f58c81607d0 2024-05-27T00:40:02.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.473+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b0012040 con 0x7f58c81607d0 2024-05-27T00:40:02.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.473+0000 7f58c57fa640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f58a806be30 0x7f58a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.473+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f58b008b8c0 con 0x7f58c81607d0 2024-05-27T00:40:02.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.474+0000 7f58c7fff640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f58a806be30 0x7f58a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.474+0000 7f58c7fff640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f58a806be30 0x7f58a806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006fd0 tx=0x7f58b8009540 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.478+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f58b0019030 con 0x7f58c81607d0 2024-05-27T00:40:02.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.652+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58c80cef10 con 0x7f58c81607d0 2024-05-27T00:40:02.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.652+0000 7f58c57fa640 1 -- 172.21.15.111:0/1282101753 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f58b0059800 con 0x7f58c81607d0 2024-05-27T00:40:02.653 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:02.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f58a806be30 msgr2=0x7f58a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f58a806be30 0x7f58a806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006fd0 tx=0x7f58b8009540 comp rx=0 tx=0).stop 2024-05-27T00:40:02.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 msgr2=0x7f58c8160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 0x7f58c8160c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58b0011920 tx=0x7f58b0011de0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 shutdown_connections 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f58a806be30 0x7f58a806e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f58c815be10 0x7f58c815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58c81607d0 0x7f58c8160c50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 --2- 172.21.15.111:0/1282101753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b4004d40 0x7f58c81590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 >> 172.21.15.111:0/1282101753 conn(0x7f58c80d42c0 msgr2=0x7f58c806b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 shutdown_connections 2024-05-27T00:40:02.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.654+0000 7f58ce3a2640 1 -- 172.21.15.111:0/1282101753 wait complete. 2024-05-27T00:40:02.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:02.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.768+0000 7f8e05fd5640 1 Processor -- start 2024-05-27T00:40:02.769 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.768+0000 7f8e05fd5640 1 -- start start 2024-05-27T00:40:02.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8e05fd5640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e000b3630 0x7f8e000b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8e05fd5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e00162560 0x7f8e000d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8e05fd5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0006cc70 con 0x7f8e000b77e0 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8e05fd5640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f8e000d1e50 con 0x7f8e000b3630 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8e05fd5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8e000d2130 con 0x7f8e00162560 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfeffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8e000b77e0 0x7f8e00161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41776/0 (socket says 172.21.15.111:41776) 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfeffd640 1 -- 172.21.15.111:0/110816367 learned_addr learned my addr 172.21.15.111:0/110816367 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfffff640 1 --2- 172.21.15.111:0/110816367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e00162560 0x7f8e000d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dff7fe640 1 --2- 172.21.15.111:0/110816367 >> v2:172.21.15.153:3300/0 conn(0x7f8e000b3630 0x7f8e000b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfe7fc640 1 -- 172.21.15.111:0/110816367 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823074457 0 0) 0x7f8e0006cc70 con 0x7f8e000b77e0 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfe7fc640 1 -- 172.21.15.111:0/110816367 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8de8003520 con 0x7f8e000b77e0 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfe7fc640 1 -- 172.21.15.111:0/110816367 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1227698805 0 0) 0x7f8de8003520 con 0x7f8e000b77e0 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfffff640 1 -- 172.21.15.111:0/110816367 >> v2:172.21.15.153:3300/0 conn(0x7f8e000b3630 msgr2=0x7f8e000b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfffff640 1 --2- 172.21.15.111:0/110816367 >> v2:172.21.15.153:3300/0 conn(0x7f8e000b3630 0x7f8e000b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfffff640 1 -- 172.21.15.111:0/110816367 >> v1:172.21.15.111:6789/0 conn(0x7f8e000b77e0 legacy=0x7f8e00161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.769+0000 7f8dfffff640 1 -- 172.21.15.111:0/110816367 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e000d2410 con 0x7f8e00162560 2024-05-27T00:40:02.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.770+0000 7f8dfffff640 1 --2- 172.21.15.111:0/110816367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e00162560 0x7f8e000d2c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009d90 tx=0x7f8df802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df349301ec6ffa05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.770+0000 7f8dfe7fc640 1 -- 172.21.15.111:0/110816367 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4001670 con 0x7f8e00162560 2024-05-27T00:40:02.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.770+0000 7f8dfe7fc640 1 -- 172.21.15.111:0/110816367 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df8007540 con 0x7f8e00162560 2024-05-27T00:40:02.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.770+0000 7f8dfe7fc640 1 -- 172.21.15.111:0/110816367 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80078f0 con 0x7f8e00162560 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 -- 172.21.15.111:0/110816367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e00162560 msgr2=0x7f8e000d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/110816367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e00162560 0x7f8e000d2c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009d90 tx=0x7f8df802a5f0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 -- 172.21.15.111:0/110816367 shutdown_connections 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/110816367 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e00162560 0x7f8e000d2c10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/110816367 >> v2:172.21.15.153:3300/0 conn(0x7f8e000b3630 0x7f8e000b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 -- 172.21.15.111:0/110816367 >> 172.21.15.111:0/110816367 conn(0x7f8e000d42c0 msgr2=0x7f8e000d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 -- 172.21.15.111:0/110816367 shutdown_connections 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.771+0000 7f8e05fd5640 1 -- 172.21.15.111:0/110816367 wait complete. 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.772+0000 7f8e05fd5640 1 Processor -- start 2024-05-27T00:40:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.772+0000 7f8e05fd5640 1 -- start start 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8e05fd5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e000b3630 0x7f8e00157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8e05fd5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e000b77e0 0x7f8e000cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8e05fd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 0x7f8e0015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8e05fd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000d32d0 con 0x7f8e00162560 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8e05fd5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8e000d3150 con 0x7f8e000b3630 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8e05fd5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8e000d3450 con 0x7f8e000b77e0 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8dfffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 0x7f8e0015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8dfffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 0x7f8e0015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42110/0 (socket says 172.21.15.111:42110) 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8dfffff640 1 -- 172.21.15.111:0/4012892882 learned_addr learned my addr 172.21.15.111:0/4012892882 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.773+0000 7f8dfeffd640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e000b77e0 0x7f8e000cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dff7fe640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e000b3630 0x7f8e00157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfffff640 1 -- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e000b77e0 msgr2=0x7f8e000cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfffff640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e000b77e0 0x7f8e000cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfffff640 1 -- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e000b3630 msgr2=0x7f8e00157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfffff640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e000b3630 0x7f8e00157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfffff640 1 -- 172.21.15.111:0/4012892882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e001578c0 con 0x7f8e00162560 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfeffd640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e000b77e0 0x7f8e000cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dff7fe640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e000b3630 0x7f8e00157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8dfffff640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 0x7f8e0015a690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009eb0 tx=0x7f8df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df803f070 con 0x7f8e00162560 2024-05-27T00:40:02.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df8031c40 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0015c8a0 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8009450 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.774+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0015ce60 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.775+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e00068680 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.775+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8df8043410 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.776+0000 7f8de7fff640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8dec06be30 0x7f8dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.776+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f8df80ac5c0 con 0x7f8e00162560 2024-05-27T00:40:02.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.776+0000 7f8dff7fe640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8dec06be30 0x7f8dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:02.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.777+0000 7f8dff7fe640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8dec06be30 0x7f8dec06e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e000ce970 tx=0x7f8df0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:02.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.780+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8df807ab50 con 0x7f8e00162560 2024-05-27T00:40:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.957+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e000cef10 con 0x7f8e00162560 2024-05-27T00:40:02.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.958+0000 7f8de7fff640 1 -- 172.21.15.111:0/4012892882 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f8df807a450 con 0x7f8e00162560 2024-05-27T00:40:02.959 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:02.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8dec06be30 msgr2=0x7f8dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8dec06be30 0x7f8dec06e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e000ce970 tx=0x7f8df0008040 comp rx=0 tx=0).stop 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 msgr2=0x7f8e0015a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 0x7f8e0015a690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009eb0 tx=0x7f8df8004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 shutdown_connections 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f8dec06be30 0x7f8dec06e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e00162560 0x7f8e0015a690 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8e000b77e0 0x7f8e000cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 --2- 172.21.15.111:0/4012892882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e000b3630 0x7f8e00157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 >> 172.21.15.111:0/4012892882 conn(0x7f8e000d42c0 msgr2=0x7f8e000b8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 shutdown_connections 2024-05-27T00:40:02.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:02.960+0000 7f8e05fd5640 1 -- 172.21.15.111:0/4012892882 wait complete. 2024-05-27T00:40:02.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:03.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.074+0000 7fa85ad89640 1 Processor -- start 2024-05-27T00:40:03.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.074+0000 7fa85ad89640 1 -- start start 2024-05-27T00:40:03.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa85ad89640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa85415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa85ad89640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540ce470 con 0x7fa85406fd90 2024-05-27T00:40:03.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa85ad89640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8540ce750 con 0x7fa8540b4ee0 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa85ad89640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa8540ce240 con 0x7fa85415c610 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa853fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa85406fd90 0x7fa85415cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41782/0 (socket says 172.21.15.111:41782) 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa853fff640 1 -- 172.21.15.111:0/4164922816 learned_addr learned my addr 172.21.15.111:0/4164922816 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907449761 0 0) 0x7fa8540ce470 con 0x7fa85406fd90 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa844003520 con 0x7fa85406fd90 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.075+0000 7fa858b25640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa85415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1183563769 0 0) 0x7fa844003520 con 0x7fa85406fd90 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 msgr2=0x7fa85415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa85415c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 >> v1:172.21.15.153:6789/0 conn(0x7fa8540b4ee0 legacy=0x7fa854165ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540cea30 con 0x7fa85406fd90 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fa840001670 con 0x7fa85406fd90 2024-05-27T00:40:03.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa85415c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 >> v1:172.21.15.111:6789/0 conn(0x7fa85406fd90 legacy=0x7fa85415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8440039e0 0x7fa844003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844004d40 0x7fa8440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa84400bb50 con 0x7fa8440039e0 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa84400bcd0 con 0x7fa844004d40 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa84400bed0 con 0x7fa85415c610 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8440039e0 0x7fa844003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa853fff640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa859326640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844004d40 0x7fa8440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 -- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 msgr2=0x7fa844004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa85415c610 0x7fa844004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 -- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844004d40 msgr2=0x7fa8440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844004d40 0x7fa8440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.076+0000 7fa858b25640 1 -- 172.21.15.111:0/4164922816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa844003520 con 0x7fa8440039e0 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa858b25640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8440039e0 0x7fa844003e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8480098a0 tx=0x7fa844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e4179c24077159 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800f040 con 0x7fa8440039e0 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84802ace0 con 0x7fa8440039e0 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa8537fe640 1 -- 172.21.15.111:0/4164922816 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848004250 con 0x7fa8440039e0 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa85ad89640 1 -- 172.21.15.111:0/4164922816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8440039e0 msgr2=0x7fa844003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa85ad89640 1 --2- 172.21.15.111:0/4164922816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8440039e0 0x7fa844003e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8480098a0 tx=0x7fa844005830 comp rx=0 tx=0).stop 2024-05-27T00:40:03.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa859326640 1 -- 172.21.15.111:0/4164922816 reap_dead start 2024-05-27T00:40:03.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa85ad89640 1 -- 172.21.15.111:0/4164922816 shutdown_connections 2024-05-27T00:40:03.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa85ad89640 1 -- 172.21.15.111:0/4164922816 >> 172.21.15.111:0/4164922816 conn(0x7fa854072a30 msgr2=0x7fa85406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa85ad89640 1 -- 172.21.15.111:0/4164922816 shutdown_connections 2024-05-27T00:40:03.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.077+0000 7fa85ad89640 1 -- 172.21.15.111:0/4164922816 wait complete. 2024-05-27T00:40:03.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 Processor -- start 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 -- start start 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440039e0 0x7fa8540acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8541483c0 0x7fa8540c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 0x7fa8540ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa85415c300 con 0x7fa8540c5d30 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540b3a40 con 0x7fa8541483c0 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.079+0000 7fa85ad89640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa854149880 con 0x7fa8440039e0 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 0x7fa8540ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 0x7fa8540ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42138/0 (socket says 172.21.15.111:42138) 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 -- 172.21.15.111:0/3983137153 learned_addr learned my addr 172.21.15.111:0/3983137153 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa853fff640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8541483c0 0x7fa8540c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa858b25640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440039e0 0x7fa8540acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 -- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440039e0 msgr2=0x7fa8540acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440039e0 0x7fa8540acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 -- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8541483c0 msgr2=0x7fa8540c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8541483c0 0x7fa8540c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 -- 172.21.15.111:0/3983137153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa854148cb0 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa859326640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 0x7fa8540ab3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8540702f0 tx=0x7fa84c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01a070 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c006d60 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa854148f50 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c003b90 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.080+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa854149510 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.081+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa854068680 con 0x7fa8540c5d30 2024-05-27T00:40:03.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.082+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa84c005070 con 0x7fa8540c5d30 2024-05-27T00:40:03.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.082+0000 7fa8517fa640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa83806be60 0x7fa83806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.082+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fa84c08a6a0 con 0x7fa8540c5d30 2024-05-27T00:40:03.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.082+0000 7fa858b25640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa83806be60 0x7fa83806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.083+0000 7fa858b25640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa83806be60 0x7fa83806e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa848033000 tx=0x7fa84802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.087+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa84c0585e0 con 0x7fa8540c5d30 2024-05-27T00:40:03.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.261+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa85406b560 con 0x7fa8540c5d30 2024-05-27T00:40:03.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.262+0000 7fa8517fa640 1 -- 172.21.15.111:0/3983137153 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa84c05c8e0 con 0x7fa8540c5d30 2024-05-27T00:40:03.263 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.263+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa83806be60 msgr2=0x7fa83806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.263+0000 7fa85ad89640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa83806be60 0x7fa83806e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa848033000 tx=0x7fa84802e000 comp rx=0 tx=0).stop 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.263+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 msgr2=0x7fa8540ab3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.263+0000 7fa85ad89640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 0x7fa8540ab3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8540702f0 tx=0x7fa84c00fd40 comp rx=0 tx=0).stop 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 shutdown_connections 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa83806be60 0x7fa83806e320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8540c5d30 0x7fa8540ab3d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8541483c0 0x7fa8540c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 --2- 172.21.15.111:0/3983137153 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa8440039e0 0x7fa8540acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 >> 172.21.15.111:0/3983137153 conn(0x7fa854072a30 msgr2=0x7fa8540b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 shutdown_connections 2024-05-27T00:40:03.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.264+0000 7fa85ad89640 1 -- 172.21.15.111:0/3983137153 wait complete. 2024-05-27T00:40:03.276 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:03.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.379+0000 7f47ca4fc640 1 Processor -- start 2024-05-27T00:40:03.380 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.379+0000 7f47ca4fc640 1 -- start start 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.379+0000 7f47ca4fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f47c40726d0 0x7f47c4072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.379+0000 7f47ca4fc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c4073020 con 0x7f47c4144000 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.379+0000 7f47ca4fc640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c40b36e0 con 0x7f47c406b390 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.379+0000 7f47ca4fc640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f47c40b42f0 con 0x7f47c40726d0 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c8a99640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f47c40726d0 0x7f47c4072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c3fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f47c4144000 0x7f47c40b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41792/0 (socket says 172.21.15.111:41792) 2024-05-27T00:40:03.381 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c3fff640 1 -- 172.21.15.111:0/439815757 learned_addr learned my addr 172.21.15.111:0/439815757 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:03.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c8a99640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f47c40726d0 0x7f47c4072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42140/0 (socket says 172.21.15.111:42140) 2024-05-27T00:40:03.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c8a99640 1 -- 172.21.15.111:0/439815757 >> v1:172.21.15.153:6789/0 conn(0x7f47c406b390 legacy=0x7f47c40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c8a99640 1 -- 172.21.15.111:0/439815757 >> v1:172.21.15.111:6789/0 conn(0x7f47c4144000 legacy=0x7f47c40b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c8a99640 1 -- 172.21.15.111:0/439815757 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c40b39c0 con 0x7f47c40726d0 2024-05-27T00:40:03.382 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.380+0000 7f47c2ffd640 1 -- 172.21.15.111:0/439815757 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320910559 0 0) 0x7f47b4001110 con 0x7f47c4144000 2024-05-27T00:40:03.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47c8a99640 1 --2- 172.21.15.111:0/439815757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c4072ae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00bfa0 tx=0x7f47bc02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea96ea430983c322 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47c2ffd640 1 -- 172.21.15.111:0/439815757 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc039070 con 0x7f47c40726d0 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47c2ffd640 1 -- 172.21.15.111:0/439815757 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc009540 con 0x7f47c40726d0 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47c2ffd640 1 -- 172.21.15.111:0/439815757 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc009910 con 0x7f47c40726d0 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 -- 172.21.15.111:0/439815757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 msgr2=0x7f47c4072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/439815757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c4072ae0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00bfa0 tx=0x7f47bc02c8b0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 -- 172.21.15.111:0/439815757 shutdown_connections 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/439815757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c4072ae0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 -- 172.21.15.111:0/439815757 >> 172.21.15.111:0/439815757 conn(0x7f47c40b1c00 msgr2=0x7f47c40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 -- 172.21.15.111:0/439815757 shutdown_connections 2024-05-27T00:40:03.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.381+0000 7f47ca4fc640 1 -- 172.21.15.111:0/439815757 wait complete. 2024-05-27T00:40:03.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.382+0000 7f47ca4fc640 1 Processor -- start 2024-05-27T00:40:03.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 -- start start 2024-05-27T00:40:03.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c406b390 0x7f47c413c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4144000 0x7f47c41e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47c40cde50 con 0x7f47c40726d0 2024-05-27T00:40:03.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f47c40b3f80 con 0x7f47c4144000 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47ca4fc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f47c40b4780 con 0x7f47c406b390 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42144/0 (socket says 172.21.15.111:42144) 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.383+0000 7f47c37fe640 1 -- 172.21.15.111:0/565275245 learned_addr learned my addr 172.21.15.111:0/565275245 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c8a99640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4144000 0x7f47c41e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c3fff640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c406b390 0x7f47c413c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c37fe640 1 -- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c406b390 msgr2=0x7f47c413c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c37fe640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c406b390 0x7f47c413c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c37fe640 1 -- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4144000 msgr2=0x7f47c41e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c37fe640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4144000 0x7f47c41e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c37fe640 1 -- 172.21.15.111:0/565275245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c41e3e00 con 0x7f47c40726d0 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c8a99640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4144000 0x7f47c41e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:03.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c3fff640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c406b390 0x7f47c413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c37fe640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c415a0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c400 tx=0x7f47ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac017070 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac0035f0 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c41e40a0 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac003a20 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.384+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41e4660 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.385+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c4068680 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.386+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47ac01a070 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.386+0000 7f47c0ff9640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47a406be60 0x7f47a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.386+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f47ac08a580 con 0x7f47c40726d0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.386+0000 7f47c3fff640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47a406be60 0x7f47a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.387+0000 7f47c3fff640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47a406be60 0x7f47a406e320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000f60 tx=0x7f47b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.390+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f47ac0010c0 con 0x7f47c40726d0 2024-05-27T00:40:03.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.562+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f47c406c920 con 0x7f47c40726d0 2024-05-27T00:40:03.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.563+0000 7f47c0ff9640 1 -- 172.21.15.111:0/565275245 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f47ac0584c0 con 0x7f47c40726d0 2024-05-27T00:40:03.564 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47a406be60 msgr2=0x7f47a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47a406be60 0x7f47a406e320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000f60 tx=0x7f47b4009040 comp rx=0 tx=0).stop 2024-05-27T00:40:03.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 msgr2=0x7f47c415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c415a0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c400 tx=0x7f47ac00c8c0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 shutdown_connections 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47a406be60 0x7f47a406e320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c4144000 0x7f47c41e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47c40726d0 0x7f47c415a0e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 --2- 172.21.15.111:0/565275245 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47c406b390 0x7f47c413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 >> 172.21.15.111:0/565275245 conn(0x7f47c40b1c00 msgr2=0x7f47c40b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 shutdown_connections 2024-05-27T00:40:03.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.565+0000 7f47ca4fc640 1 -- 172.21.15.111:0/565275245 wait complete. 2024-05-27T00:40:03.577 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:03.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.676+0000 7f97b3929640 1 Processor -- start 2024-05-27T00:40:03.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.676+0000 7f97b3929640 1 -- start start 2024-05-27T00:40:03.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b3929640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97ac15c610 0x7f97ac15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b3929640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ac0ce470 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b3929640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ac0ce750 con 0x7f97ac06fd90 2024-05-27T00:40:03.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b3929640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f97ac0ce240 con 0x7f97ac15c610 2024-05-27T00:40:03.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b16c5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97ac15c610 0x7f97ac15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b1ec6640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f97ac0b4ee0 0x7f97ac165ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41806/0 (socket says 172.21.15.111:41806) 2024-05-27T00:40:03.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b1ec6640 1 -- 172.21.15.111:0/3336798923 learned_addr learned my addr 172.21.15.111:0/3336798923 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:03.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b16c5640 1 -- 172.21.15.111:0/3336798923 >> v1:172.21.15.153:6789/0 conn(0x7f97ac06fd90 legacy=0x7f97ac15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b16c5640 1 -- 172.21.15.111:0/3336798923 >> v1:172.21.15.111:6789/0 conn(0x7f97ac0b4ee0 legacy=0x7f97ac165ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.677+0000 7f97b16c5640 1 -- 172.21.15.111:0/3336798923 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac0cea30 con 0x7f97ac15c610 2024-05-27T00:40:03.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.678+0000 7f97b16c5640 1 --2- 172.21.15.111:0/3336798923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac15c610 0x7f97ac15c9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f979c009d90 tx=0x7f979c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5cf61558cf974b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.678+0000 7f97abfff640 1 -- 172.21.15.111:0/3336798923 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c031600 con 0x7f97ac15c610 2024-05-27T00:40:03.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.678+0000 7f97abfff640 1 -- 172.21.15.111:0/3336798923 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c0317a0 con 0x7f97ac15c610 2024-05-27T00:40:03.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.678+0000 7f97abfff640 1 -- 172.21.15.111:0/3336798923 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c031b70 con 0x7f97ac15c610 2024-05-27T00:40:03.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 -- 172.21.15.111:0/3336798923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac15c610 msgr2=0x7f97ac15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 --2- 172.21.15.111:0/3336798923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac15c610 0x7f97ac15c9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f979c009d90 tx=0x7f979c02a610 comp rx=0 tx=0).stop 2024-05-27T00:40:03.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 -- 172.21.15.111:0/3336798923 shutdown_connections 2024-05-27T00:40:03.688 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 --2- 172.21.15.111:0/3336798923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac15c610 0x7f97ac15c9f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 -- 172.21.15.111:0/3336798923 >> 172.21.15.111:0/3336798923 conn(0x7f97ac072a30 msgr2=0x7f97ac06bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 -- 172.21.15.111:0/3336798923 shutdown_connections 2024-05-27T00:40:03.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.679+0000 7f97b3929640 1 -- 172.21.15.111:0/3336798923 wait complete. 2024-05-27T00:40:03.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.680+0000 7f97b3929640 1 Processor -- start 2024-05-27T00:40:03.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.680+0000 7f97b3929640 1 -- start start 2024-05-27T00:40:03.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac06fd90 0x7f97ac145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 0x7f97ac152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97ac15c610 0x7f97ac13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac15c300 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac0b3a40 con 0x7f97ac06fd90 2024-05-27T00:40:03.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac153360 con 0x7f97ac15c610 2024-05-27T00:40:03.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 0x7f97ac152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 0x7f97ac152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42164/0 (socket says 172.21.15.111:42164) 2024-05-27T00:40:03.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 -- 172.21.15.111:0/100156293 learned_addr learned my addr 172.21.15.111:0/100156293 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:03.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 -- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97ac15c610 msgr2=0x7f97ac13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97ac15c610 0x7f97ac13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 -- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac06fd90 msgr2=0x7f97ac145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac06fd90 0x7f97ac145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 -- 172.21.15.111:0/100156293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac153590 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b0ec4640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 0x7f97ac152e20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f97a000b3e0 tx=0x7f97a000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0015020 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a0011850 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.681+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac153870 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.682+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0011c40 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.682+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac153e30 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.683+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ac068680 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.683+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97a0018020 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.684+0000 7f97a9ffb640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f979806be60 0x7f979806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.684+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f97a008b150 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.684+0000 7f97b16c5640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f979806be60 0x7f979806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.684+0000 7f97b16c5640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f979806be60 0x7f979806e320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.688+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f97a001f020 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.861+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97ac06d530 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.862+0000 7f97a9ffb640 1 -- 172.21.15.111:0/100156293 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f97a00172c0 con 0x7f97ac0b4ee0 2024-05-27T00:40:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:03.863 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:03.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f979806be60 msgr2=0x7f979806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f979806be60 0x7f979806e320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c02e000 comp rx=0 tx=0).stop 2024-05-27T00:40:03.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 msgr2=0x7f97ac152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 0x7f97ac152e20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f97a000b3e0 tx=0x7f97a000b8a0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 shutdown_connections 2024-05-27T00:40:03.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f979806be60 0x7f979806e320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f97ac15c610 0x7f97ac13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97ac0b4ee0 0x7f97ac152e20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 --2- 172.21.15.111:0/100156293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac06fd90 0x7f97ac145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 >> 172.21.15.111:0/100156293 conn(0x7f97ac072a30 msgr2=0x7f97ac06ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 shutdown_connections 2024-05-27T00:40:03.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.864+0000 7f97b3929640 1 -- 172.21.15.111:0/100156293 wait complete. 2024-05-27T00:40:03.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:03.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:03.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.979+0000 7f5bf78ba640 1 Processor -- start 2024-05-27T00:40:03.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.980+0000 7f5bf78ba640 1 -- start start 2024-05-27T00:40:03.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.980+0000 7f5bf78ba640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 0x7f5bf00cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.980+0000 7f5bf78ba640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf00b36e0 con 0x7f5bf00cdc80 2024-05-27T00:40:03.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.980+0000 7f5bf78ba640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf00b39c0 con 0x7f5bf0144000 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.980+0000 7f5bf78ba640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf00b42f0 con 0x7f5bf006b390 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5bf5e57640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5bf00cdc80 0x7f5bf00726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41818/0 (socket says 172.21.15.111:41818) 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5bf5e57640 1 -- 172.21.15.111:0/3582909053 learned_addr learned my addr 172.21.15.111:0/3582909053 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5bf4e55640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 0x7f5bf00cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360787567 0 0) 0x7f5bf00b36e0 con 0x7f5bf00cdc80 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bdc003520 con 0x7f5bf00cdc80 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1510269262 0 0) 0x7f5bdc003520 con 0x7f5bf00cdc80 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 msgr2=0x7f5bf00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 0x7f5bf00cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 >> v1:172.21.15.153:6789/0 conn(0x7f5bf0144000 legacy=0x7f5bf00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf00b3ca0 con 0x7f5bf00cdc80 2024-05-27T00:40:03.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f5bec001670 con 0x7f5bf00cdc80 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 >> v1:172.21.15.111:6789/0 conn(0x7f5bf00cdc80 legacy=0x7f5bf00726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 0x7f5bdc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bdc003f60 0x7f5bdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00b42f0 con 0x7f5bdc004d40 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc00baa0 con 0x7f5bdc003f60 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc00bca0 con 0x7f5bf006b390 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5bf5e57640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5bf5656640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 0x7f5bdc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.981+0000 7f5bf4e55640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bdc003f60 0x7f5bdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5bf5e57640 1 -- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 msgr2=0x7f5bdc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5bf5e57640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf006b390 0x7f5bdc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5bf5e57640 1 -- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bdc003f60 msgr2=0x7f5bdc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5bf5e57640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bdc003f60 0x7f5bdc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5bf5e57640 1 -- 172.21.15.111:0/3582909053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc003520 con 0x7f5bdc004d40 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5bf5e57640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bdc0051c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5bf006ce00 tx=0x7f5bdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39173ad1e3d13490 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec007d10 con 0x7f5bdc004d40 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec005ad0 con 0x7f5bdc004d40 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.982+0000 7f5be7fff640 1 -- 172.21.15.111:0/3582909053 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec007070 con 0x7f5bdc004d40 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf78ba640 1 -- 172.21.15.111:0/3582909053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 msgr2=0x7f5bdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/3582909053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bdc0051c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5bf006ce00 tx=0x7f5bdc005830 comp rx=0 tx=0).stop 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf5e57640 1 -- 172.21.15.111:0/3582909053 reap_dead start 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf78ba640 1 -- 172.21.15.111:0/3582909053 shutdown_connections 2024-05-27T00:40:03.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf78ba640 1 -- 172.21.15.111:0/3582909053 >> 172.21.15.111:0/3582909053 conn(0x7f5bf00b1c00 msgr2=0x7f5bf00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:03.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf78ba640 1 -- 172.21.15.111:0/3582909053 shutdown_connections 2024-05-27T00:40:03.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.983+0000 7f5bf78ba640 1 -- 172.21.15.111:0/3582909053 wait complete. 2024-05-27T00:40:03.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.984+0000 7f5bf78ba640 1 Processor -- start 2024-05-27T00:40:03.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.984+0000 7f5bf78ba640 1 -- start start 2024-05-27T00:40:03.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf78ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bf0142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf78ba640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf01eeba0 0x7f5bf01ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf78ba640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bf01f2fd0 0x7f5bf01f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf78ba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf0072f60 con 0x7f5bdc004d40 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf78ba640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf0072de0 con 0x7f5bf01f2fd0 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf78ba640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf00cd740 con 0x7f5bf01eeba0 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bf0142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bf0142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42168/0 (socket says 172.21.15.111:42168) 2024-05-27T00:40:03.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 -- 172.21.15.111:0/2307155144 learned_addr learned my addr 172.21.15.111:0/2307155144 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf4e55640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf01eeba0 0x7f5bf01ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5e57640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bf01f2fd0 0x7f5bf01f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 -- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf01eeba0 msgr2=0x7f5bf01ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf01eeba0 0x7f5bf01ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 -- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bf01f2fd0 msgr2=0x7f5bf01f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bf01f2fd0 0x7f5bf01f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 -- 172.21.15.111:0/2307155144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf015b670 con 0x7f5bdc004d40 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5bf5656640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bf0142dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5be800fbc0 tx=0x7f5be800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.985+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be800dc30 con 0x7f5bdc004d40 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.986+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be8018070 con 0x7f5bdc004d40 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.986+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf015b8e0 con 0x7f5bdc004d40 2024-05-27T00:40:03.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.986+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003970 con 0x7f5bdc004d40 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.986+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf01ef560 con 0x7f5bdc004d40 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.987+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf0068680 con 0x7f5bdc004d40 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.987+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5be8003b10 con 0x7f5bdc004d40 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.987+0000 7f5be5ffb640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5bd406be30 0x7f5bd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.988+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f5be808b420 con 0x7f5bdc004d40 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.988+0000 7f5bf4e55640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5bd406be30 0x7f5bd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.988+0000 7f5bf4e55640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5bd406be30 0x7f5bd406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0072d30 tx=0x7f5be0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:03.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:03.992+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5be8059a40 con 0x7f5bdc004d40 2024-05-27T00:40:04.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.168+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bf00b3220 con 0x7f5bdc004d40 2024-05-27T00:40:04.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.169+0000 7f5be5ffb640 1 -- 172.21.15.111:0/2307155144 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f5be8059340 con 0x7f5bdc004d40 2024-05-27T00:40:04.170 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5bd406be30 msgr2=0x7f5bd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5bd406be30 0x7f5bd406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0072d30 tx=0x7f5be0008040 comp rx=0 tx=0).stop 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 msgr2=0x7f5bf0142dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bf0142dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5be800fbc0 tx=0x7f5be800d520 comp rx=0 tx=0).stop 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 shutdown_connections 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5bd406be30 0x7f5bd406e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5bf01f2fd0 0x7f5bf01f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5bf01eeba0 0x7f5bf01ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 --2- 172.21.15.111:0/2307155144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5bdc004d40 0x7f5bf0142dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 >> 172.21.15.111:0/2307155144 conn(0x7f5bf00b1c00 msgr2=0x7f5bf006bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:04.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 shutdown_connections 2024-05-27T00:40:04.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.171+0000 7f5bf78ba640 1 -- 172.21.15.111:0/2307155144 wait complete. 2024-05-27T00:40:04.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:04.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.284+0000 7fc58b676640 1 Processor -- start 2024-05-27T00:40:04.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.284+0000 7fc58b676640 1 -- start start 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc58b676640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc58413f340 0x7fc5840cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc58b676640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc584142650 con 0x7fc5840cce00 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc58b676640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5840cebc0 con 0x7fc58413ef60 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc58b676640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc5840cd270 con 0x7fc58413f340 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc589c13640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc5840cce00 0x7fc584141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41826/0 (socket says 172.21.15.111:41826) 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc589c13640 1 -- 172.21.15.111:0/1972546628 learned_addr learned my addr 172.21.15.111:0/1972546628 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921550979 0 0) 0x7fc584142650 con 0x7fc5840cce00 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc570003520 con 0x7fc5840cce00 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.285+0000 7fc588c11640 1 --2- 172.21.15.111:0/1972546628 >> v2:172.21.15.153:3300/0 conn(0x7fc58413f340 0x7fc5840cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333365297 0 0) 0x7fc570003520 con 0x7fc5840cce00 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 >> v2:172.21.15.153:3300/0 conn(0x7fc58413f340 msgr2=0x7fc5840cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:04.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 --2- 172.21.15.111:0/1972546628 >> v2:172.21.15.153:3300/0 conn(0x7fc58413f340 0x7fc5840cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 >> v1:172.21.15.153:6789/0 conn(0x7fc58413ef60 legacy=0x7fc5840cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5840ceea0 con 0x7fc5840cce00 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fc57c0019f0 con 0x7fc5840cce00 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 >> v1:172.21.15.111:6789/0 conn(0x7fc5840cce00 legacy=0x7fc584141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5840cd270 con 0x7fc5700039b0 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc57000c060 con 0x7fc570004800 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc57000c260 con 0x7fc5700051c0 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589412640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc588c11640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589c13640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589412640 1 -- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700051c0 msgr2=0x7fc57000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589412640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589412640 1 -- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc570004800 msgr2=0x7fc570004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589412640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589412640 1 -- 172.21.15.111:0/1972546628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc570003520 con 0x7fc5700039b0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589c13640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.286+0000 7fc589c13640 1 -- 172.21.15.111:0/1972546628 reap_dead start 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc589412640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc580001750 tx=0x7fc570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d0f3827d02d23c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580001370 con 0x7fc5700039b0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5800052d0 con 0x7fc5700039b0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc57bfff640 1 -- 172.21.15.111:0/1972546628 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58002abc0 con 0x7fc5700039b0 2024-05-27T00:40:04.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc58b676640 1 -- 172.21.15.111:0/1972546628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5700039b0 msgr2=0x7fc570003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc58b676640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc580001750 tx=0x7fc570005830 comp rx=0 tx=0).stop 2024-05-27T00:40:04.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc58b676640 1 -- 172.21.15.111:0/1972546628 shutdown_connections 2024-05-27T00:40:04.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc58b676640 1 --2- 172.21.15.111:0/1972546628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.287+0000 7fc58b676640 1 -- 172.21.15.111:0/1972546628 >> 172.21.15.111:0/1972546628 conn(0x7fc5840b1c00 msgr2=0x7fc5840b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:04.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.288+0000 7fc58b676640 1 -- 172.21.15.111:0/1972546628 shutdown_connections 2024-05-27T00:40:04.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.288+0000 7fc58b676640 1 -- 172.21.15.111:0/1972546628 wait complete. 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 Processor -- start 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 -- start start 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700039b0 0x7fc584156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc584159900 0x7fc5841f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 0x7fc584072750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5840cc8c0 con 0x7fc5841f51a0 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5840cf210 con 0x7fc584159900 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.289+0000 7fc58b676640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc5840cca40 con 0x7fc5700039b0 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 0x7fc584072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 0x7fc584072750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42190/0 (socket says 172.21.15.111:42190) 2024-05-27T00:40:04.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 -- 172.21.15.111:0/2277592206 learned_addr learned my addr 172.21.15.111:0/2277592206 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc588c11640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc584159900 0x7fc5841f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589412640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700039b0 0x7fc584156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 -- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700039b0 msgr2=0x7fc584156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700039b0 0x7fc584156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 -- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc584159900 msgr2=0x7fc5841f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc584159900 0x7fc5841f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 -- 172.21.15.111:0/2277592206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5841f5b10 con 0x7fc5841f51a0 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589412640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700039b0 0x7fc584156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:04.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc588c11640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc584159900 0x7fc5841f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc589c13640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 0x7fc584072750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc58406f770 tx=0x7fc57c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00eda0 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c018070 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c013640 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841f5df0 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.290+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841570b0 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.291+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5841572b0 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.292+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc57c0137e0 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.292+0000 7fc579ffb640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc56806be60 0x7fc56806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.292+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fc57c08b4f0 con 0x7fc5841f51a0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.292+0000 7fc589412640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc56806be60 0x7fc56806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.293+0000 7fc589412640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc56806be60 0x7fc56806e320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc580032000 tx=0x7fc58002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.296+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc5841572b0 con 0x7fc5841f51a0 2024-05-27T00:40:04.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.473+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc584068680 con 0x7fc5841f51a0 2024-05-27T00:40:04.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.474+0000 7fc579ffb640 1 -- 172.21.15.111:0/2277592206 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc57c059b10 con 0x7fc5841f51a0 2024-05-27T00:40:04.475 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:04.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc56806be60 msgr2=0x7fc56806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc56806be60 0x7fc56806e320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc580032000 tx=0x7fc58002e000 comp rx=0 tx=0).stop 2024-05-27T00:40:04.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 msgr2=0x7fc584072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.477 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 0x7fc584072750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc58406f770 tx=0x7fc57c00e530 comp rx=0 tx=0).stop 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 shutdown_connections 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc56806be60 0x7fc56806e320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5841f51a0 0x7fc584072750 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc584159900 0x7fc5841f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 --2- 172.21.15.111:0/2277592206 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5700039b0 0x7fc584156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 >> 172.21.15.111:0/2277592206 conn(0x7fc5840b1c00 msgr2=0x7fc58415ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 shutdown_connections 2024-05-27T00:40:04.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.476+0000 7fc58b676640 1 -- 172.21.15.111:0/2277592206 wait complete. 2024-05-27T00:40:04.488 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:04.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.590+0000 7f4a6af63640 1 Processor -- start 2024-05-27T00:40:04.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.590+0000 7f4a6af63640 1 -- start start 2024-05-27T00:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a6af63640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4a64144000 0x7f4a640b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a6af63640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4a6406b390 0x7f4a640cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a6af63640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a640b3440 con 0x7f4a640cdc80 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a6af63640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640b42f0 con 0x7f4a6406b390 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a6af63640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640b36e0 con 0x7f4a64144000 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a63fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4a6406b390 0x7f4a640cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a69500640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4a640cdc80 0x7f4a640726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41832/0 (socket says 172.21.15.111:41832) 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a69500640 1 -- 172.21.15.111:0/2383344886 learned_addr learned my addr 172.21.15.111:0/2383344886 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a63fff640 1 -- 172.21.15.111:0/2383344886 >> v2:172.21.15.153:3300/0 conn(0x7f4a64144000 msgr2=0x7f4a640b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a63fff640 1 --2- 172.21.15.111:0/2383344886 >> v2:172.21.15.153:3300/0 conn(0x7f4a64144000 0x7f4a640b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.591+0000 7f4a63fff640 1 -- 172.21.15.111:0/2383344886 >> v1:172.21.15.111:6789/0 conn(0x7f4a640cdc80 legacy=0x7f4a640726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.592+0000 7f4a63fff640 1 -- 172.21.15.111:0/2383344886 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640b39c0 con 0x7f4a6406b390 2024-05-27T00:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.592+0000 7f4a637fe640 1 -- 172.21.15.111:0/2383344886 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763236536 0 0) 0x7f4a640b3440 con 0x7f4a640cdc80 2024-05-27T00:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.592+0000 7f4a63fff640 1 --2- 172.21.15.111:0/2383344886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a6406b390 0x7f4a640cd740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4a640726a0 tx=0x7f4a5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21a11b3529abc44 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.592+0000 7f4a637fe640 1 -- 172.21.15.111:0/2383344886 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58037070 con 0x7f4a6406b390 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.592+0000 7f4a637fe640 1 -- 172.21.15.111:0/2383344886 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a58032440 con 0x7f4a6406b390 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.592+0000 7f4a637fe640 1 -- 172.21.15.111:0/2383344886 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58032810 con 0x7f4a6406b390 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 -- 172.21.15.111:0/2383344886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a6406b390 msgr2=0x7f4a640cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2383344886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a6406b390 0x7f4a640cd740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4a640726a0 tx=0x7f4a5802a1a0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 -- 172.21.15.111:0/2383344886 shutdown_connections 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2383344886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a6406b390 0x7f4a640cd740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2383344886 >> v2:172.21.15.153:3300/0 conn(0x7f4a64144000 0x7f4a640b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 -- 172.21.15.111:0/2383344886 >> 172.21.15.111:0/2383344886 conn(0x7f4a640b1c00 msgr2=0x7f4a640b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 -- 172.21.15.111:0/2383344886 shutdown_connections 2024-05-27T00:40:04.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.593+0000 7f4a6af63640 1 -- 172.21.15.111:0/2383344886 wait complete. 2024-05-27T00:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.594+0000 7f4a6af63640 1 Processor -- start 2024-05-27T00:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.594+0000 7f4a6af63640 1 -- start start 2024-05-27T00:40:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a6af63640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a6406b390 0x7f4a64157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a6af63640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 0x7f4a640ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a6af63640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a64144000 0x7f4a640cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a6af63640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64072de0 con 0x7f4a640cdc80 2024-05-27T00:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a6af63640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b3f80 con 0x7f4a64144000 2024-05-27T00:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a6af63640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4a64072f60 con 0x7f4a6406b390 2024-05-27T00:40:04.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 0x7f4a640ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 0x7f4a640ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42210/0 (socket says 172.21.15.111:42210) 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 -- 172.21.15.111:0/2978493278 learned_addr learned my addr 172.21.15.111:0/2978493278 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a68cff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a6406b390 0x7f4a64157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a69500640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a64144000 0x7f4a640cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 -- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a6406b390 msgr2=0x7f4a64157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a6406b390 0x7f4a64157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 -- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a64144000 msgr2=0x7f4a640cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a64144000 0x7f4a640cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 -- 172.21.15.111:0/2978493278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641f2060 con 0x7f4a640cdc80 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a68cff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a6406b390 0x7f4a64157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:04.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a69500640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a64144000 0x7f4a640cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.595+0000 7f4a63fff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 0x7f4a640ceb10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009890 tx=0x7f4a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.596+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58037040 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.596+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a58004230 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.596+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a641f2340 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.596+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58032e20 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.596+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641f2900 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.597+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a64068680 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.597+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a58030770 con 0x7f4a640cdc80 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.597+0000 7f4a617fa640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a5006be30 0x7f4a5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.598+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f4a580ac580 con 0x7f4a640cdc80 2024-05-27T00:40:04.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.598+0000 7f4a68cff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a5006be30 0x7f4a5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.598+0000 7f4a68cff640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a5006be30 0x7f4a5006e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a54007eb0 tx=0x7f4a540073c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.602+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4a5807ac30 con 0x7f4a640cdc80 2024-05-27T00:40:04.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.775+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a640cc640 con 0x7f4a640cdc80 2024-05-27T00:40:04.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.775+0000 7f4a617fa640 1 -- 172.21.15.111:0/2978493278 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f4a5807a530 con 0x7f4a640cdc80 2024-05-27T00:40:04.777 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:04.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a5006be30 msgr2=0x7f4a5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a5006be30 0x7f4a5006e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a54007eb0 tx=0x7f4a540073c0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 msgr2=0x7f4a640ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 0x7f4a640ceb10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009890 tx=0x7f4a580046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 shutdown_connections 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4a5006be30 0x7f4a5006e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a64144000 0x7f4a640cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a640cdc80 0x7f4a640ceb10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 --2- 172.21.15.111:0/2978493278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4a6406b390 0x7f4a64157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 >> 172.21.15.111:0/2978493278 conn(0x7f4a640b1c00 msgr2=0x7f4a641376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:04.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 shutdown_connections 2024-05-27T00:40:04.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.777+0000 7f4a6af63640 1 -- 172.21.15.111:0/2978493278 wait complete. 2024-05-27T00:40:04.789 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:04.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.891+0000 7f9946838640 1 Processor -- start 2024-05-27T00:40:04.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.891+0000 7f9946838640 1 -- start start 2024-05-27T00:40:04.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9946838640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9940144000 0x7f99400b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9946838640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f994006b390 0x7f99400cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9946838640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99400726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9946838640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f99400b42f0 con 0x7f994006b390 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9946838640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f99400b3440 con 0x7f9940144000 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9946838640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f99400b36e0 con 0x7f99400cdc80 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9937fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f994006b390 0x7f99400cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9937fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f994006b390 0x7f99400cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42218/0 (socket says 172.21.15.111:42218) 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.892+0000 7f9937fff640 1 -- 172.21.15.111:0/2820035701 learned_addr learned my addr 172.21.15.111:0/2820035701 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9944dd5640 1 --2- 172.21.15.111:0/2820035701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99400726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f993ffff640 1 --2- 172.21.15.111:0/2820035701 >> v2:172.21.15.153:3300/0 conn(0x7f9940144000 0x7f99400b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9937fff640 1 -- 172.21.15.111:0/2820035701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 msgr2=0x7f99400726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9937fff640 1 --2- 172.21.15.111:0/2820035701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99400726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9937fff640 1 -- 172.21.15.111:0/2820035701 >> v2:172.21.15.153:3300/0 conn(0x7f9940144000 msgr2=0x7f99400b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9937fff640 1 --2- 172.21.15.111:0/2820035701 >> v2:172.21.15.153:3300/0 conn(0x7f9940144000 0x7f99400b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9937fff640 1 -- 172.21.15.111:0/2820035701 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99400b39c0 con 0x7f994006b390 2024-05-27T00:40:04.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f993ffff640 1 --2- 172.21.15.111:0/2820035701 >> v2:172.21.15.153:3300/0 conn(0x7f9940144000 0x7f99400b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:04.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.893+0000 7f9937fff640 1 --2- 172.21.15.111:0/2820035701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f994006b390 0x7f99400cd740 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f99400726a0 tx=0x7f992802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba0e611f23e6741 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f993f7fe640 1 -- 172.21.15.111:0/2820035701 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928032790 con 0x7f994006b390 2024-05-27T00:40:04.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f993f7fe640 1 -- 172.21.15.111:0/2820035701 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9928038070 con 0x7f994006b390 2024-05-27T00:40:04.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f993f7fe640 1 -- 172.21.15.111:0/2820035701 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928032b50 con 0x7f994006b390 2024-05-27T00:40:04.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 -- 172.21.15.111:0/2820035701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f994006b390 msgr2=0x7f99400cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 --2- 172.21.15.111:0/2820035701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f994006b390 0x7f99400cd740 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f99400726a0 tx=0x7f992802c890 comp rx=0 tx=0).stop 2024-05-27T00:40:04.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 -- 172.21.15.111:0/2820035701 shutdown_connections 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 --2- 172.21.15.111:0/2820035701 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99400726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 --2- 172.21.15.111:0/2820035701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f994006b390 0x7f99400cd740 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 --2- 172.21.15.111:0/2820035701 >> v2:172.21.15.153:3300/0 conn(0x7f9940144000 0x7f99400b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 -- 172.21.15.111:0/2820035701 >> 172.21.15.111:0/2820035701 conn(0x7f99400b1c00 msgr2=0x7f99400b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 -- 172.21.15.111:0/2820035701 shutdown_connections 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.894+0000 7f9946838640 1 -- 172.21.15.111:0/2820035701 wait complete. 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.895+0000 7f9946838640 1 Processor -- start 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 -- start start 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f994006b390 0x7f99400a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99401598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 0x7f9940159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9940072c10 con 0x7f9940144000 2024-05-27T00:40:04.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f99400b3f80 con 0x7f994006b390 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9946838640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9940072d90 con 0x7f99400cdc80 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9944dd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 0x7f9940159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9944dd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 0x7f9940159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42228/0 (socket says 172.21.15.111:42228) 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.896+0000 7f9944dd5640 1 -- 172.21.15.111:0/4127397976 learned_addr learned my addr 172.21.15.111:0/4127397976 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f993ffff640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f994006b390 0x7f99400a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9937fff640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99401598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9944dd5640 1 -- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 msgr2=0x7f99401598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9944dd5640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99401598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9944dd5640 1 -- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f994006b390 msgr2=0x7f99400a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9944dd5640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f994006b390 0x7f99400a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9944dd5640 1 -- 172.21.15.111:0/4127397976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99400a9940 con 0x7f9940144000 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f993ffff640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f994006b390 0x7f99400a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:04.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9944dd5640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 0x7f9940159e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9938007e60 tx=0x7f993800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938014620 con 0x7f9940144000 2024-05-27T00:40:04.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9938019070 con 0x7f9940144000 2024-05-27T00:40:04.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99401e3490 con 0x7f9940144000 2024-05-27T00:40:04.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938014a20 con 0x7f9940144000 2024-05-27T00:40:04.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.897+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401e3a50 con 0x7f9940144000 2024-05-27T00:40:04.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.898+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9940068680 con 0x7f9940144000 2024-05-27T00:40:04.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.898+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9938014bc0 con 0x7f9940144000 2024-05-27T00:40:04.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.899+0000 7f993d7fa640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f992006be60 0x7f992006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:04.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.899+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f993808baf0 con 0x7f9940144000 2024-05-27T00:40:04.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.899+0000 7f993ffff640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f992006be60 0x7f992006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:04.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.899+0000 7f993ffff640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f992006be60 0x7f992006e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9930006fd0 tx=0x7f9930008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:04.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:04.903+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f993805a080 con 0x7f9940144000 2024-05-27T00:40:05.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.076+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f99400ced10 con 0x7f9940144000 2024-05-27T00:40:05.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.076+0000 7f993d7fa640 1 -- 172.21.15.111:0/4127397976 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9938059980 con 0x7f9940144000 2024-05-27T00:40:05.077 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f992006be60 msgr2=0x7f992006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f992006be60 0x7f992006e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9930006fd0 tx=0x7f9930008040 comp rx=0 tx=0).stop 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 msgr2=0x7f9940159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 0x7f9940159e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9938007e60 tx=0x7f993800e4b0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 shutdown_connections 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f992006be60 0x7f992006e320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9940144000 0x7f9940159e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f99400cdc80 0x7f99401598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 --2- 172.21.15.111:0/4127397976 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f994006b390 0x7f99400a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.078+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 >> 172.21.15.111:0/4127397976 conn(0x7f99400b1c00 msgr2=0x7f99400cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.079+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 shutdown_connections 2024-05-27T00:40:05.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.079+0000 7f9946838640 1 -- 172.21.15.111:0/4127397976 wait complete. 2024-05-27T00:40:05.107 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-05-27T00:40:05.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:05.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.205+0000 7fb1110ee640 1 Processor -- start 2024-05-27T00:40:05.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.205+0000 7fb1110ee640 1 -- start start 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb1110ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb10c144000 0x7fb10c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb1110ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10c0b3440 con 0x7fb10c0726d0 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb1110ee640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10c0b36e0 con 0x7fb10c06b390 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb1110ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c0b42f0 con 0x7fb10c144000 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb10ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb10c144000 0x7fb10c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb10b59e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb10c0726d0 0x7fb10c072ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41846/0 (socket says 172.21.15.111:41846) 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.206+0000 7fb10b59e640 1 -- 172.21.15.111:0/2314932591 learned_addr learned my addr 172.21.15.111:0/2314932591 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.207+0000 7fb10ad9d640 1 -- 172.21.15.111:0/2314932591 >> v1:172.21.15.153:6789/0 conn(0x7fb10c06b390 legacy=0x7fb10c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.207+0000 7fb10ad9d640 1 -- 172.21.15.111:0/2314932591 >> v1:172.21.15.111:6789/0 conn(0x7fb10c0726d0 legacy=0x7fb10c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.207+0000 7fb10ad9d640 1 -- 172.21.15.111:0/2314932591 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c0b39c0 con 0x7fb10c144000 2024-05-27T00:40:05.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.207+0000 7fb109d9b640 1 -- 172.21.15.111:0/2314932591 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823819974 0 0) 0x7fb10c0b3440 con 0x7fb10c0726d0 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.207+0000 7fb10ad9d640 1 --2- 172.21.15.111:0/2314932591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c0b2f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0b4470 tx=0x7fb0fc02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5d36ff301a17e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb109d9b640 1 -- 172.21.15.111:0/2314932591 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc032750 con 0x7fb10c144000 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb109d9b640 1 -- 172.21.15.111:0/2314932591 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc030050 con 0x7fb10c144000 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb109d9b640 1 -- 172.21.15.111:0/2314932591 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc032b10 con 0x7fb10c144000 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 -- 172.21.15.111:0/2314932591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 msgr2=0x7fb10c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 --2- 172.21.15.111:0/2314932591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c0b2f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0b4470 tx=0x7fb0fc02c890 comp rx=0 tx=0).stop 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 -- 172.21.15.111:0/2314932591 shutdown_connections 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 --2- 172.21.15.111:0/2314932591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c0b2f00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 -- 172.21.15.111:0/2314932591 >> 172.21.15.111:0/2314932591 conn(0x7fb10c0b1c00 msgr2=0x7fb10c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:05.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 -- 172.21.15.111:0/2314932591 shutdown_connections 2024-05-27T00:40:05.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.208+0000 7fb1110ee640 1 -- 172.21.15.111:0/2314932591 wait complete. 2024-05-27T00:40:05.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 Processor -- start 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 -- start start 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb10c06b390 0x7fb10c142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10c0726d0 0x7fb10c0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c1f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c0cde50 con 0x7fb10c144000 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c0b3f80 con 0x7fb10c06b390 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb1110ee640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c0b4780 con 0x7fb10c0726d0 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb10b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c1f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb10b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c1f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42246/0 (socket says 172.21.15.111:42246) 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.210+0000 7fb10b59e640 1 -- 172.21.15.111:0/558167060 learned_addr learned my addr 172.21.15.111:0/558167060 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10ad9d640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb10c06b390 0x7fb10c142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10a59c640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10c0726d0 0x7fb10c0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10b59e640 1 -- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10c0726d0 msgr2=0x7fb10c0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10b59e640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10c0726d0 0x7fb10c0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10b59e640 1 -- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb10c06b390 msgr2=0x7fb10c142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10b59e640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb10c06b390 0x7fb10c142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10b59e640 1 -- 172.21.15.111:0/558167060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c0c7190 con 0x7fb10c144000 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb10b59e640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c1f32e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb10400c7a0 tx=0x7fb10400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104018070 con 0x7fb10c144000 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb10400a640 con 0x7fb10c144000 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c0acf70 con 0x7fb10c144000 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10400aab0 con 0x7fb10c144000 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.211+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c142b90 con 0x7fb10c144000 2024-05-27T00:40:05.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.212+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10c068680 con 0x7fb10c144000 2024-05-27T00:40:05.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.213+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb10401b070 con 0x7fb10c144000 2024-05-27T00:40:05.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.213+0000 7fb1037fe640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb0ec06be30 0x7fb0ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.213+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fb10408a8d0 con 0x7fb10c144000 2024-05-27T00:40:05.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.213+0000 7fb10ad9d640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb0ec06be30 0x7fb0ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.214+0000 7fb10ad9d640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb0ec06be30 0x7fb0ec06e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc037000 tx=0x7fb0fc032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.217+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb104058e60 con 0x7fb10c144000 2024-05-27T00:40:05.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.390+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb10c0b0820 con 0x7fb10c144000 2024-05-27T00:40:05.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.391+0000 7fb1037fe640 1 -- 172.21.15.111:0/558167060 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7fb104058760 con 0x7fb10c144000 2024-05-27T00:40:05.392 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:05.392 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:05.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb0ec06be30 msgr2=0x7fb0ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb0ec06be30 0x7fb0ec06e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc037000 tx=0x7fb0fc032000 comp rx=0 tx=0).stop 2024-05-27T00:40:05.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 msgr2=0x7fb10c1f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c1f32e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb10400c7a0 tx=0x7fb10400cc60 comp rx=0 tx=0).stop 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 shutdown_connections 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb0ec06be30 0x7fb0ec06e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb10c144000 0x7fb10c1f32e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb10c0726d0 0x7fb10c0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 --2- 172.21.15.111:0/558167060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb10c06b390 0x7fb10c142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 >> 172.21.15.111:0/558167060 conn(0x7fb10c0b1c00 msgr2=0x7fb10c0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 shutdown_connections 2024-05-27T00:40:05.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.393+0000 7fb1110ee640 1 -- 172.21.15.111:0/558167060 wait complete. 2024-05-27T00:40:05.406 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:05.406 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:05.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.508+0000 7f2978d63640 1 Processor -- start 2024-05-27T00:40:05.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.508+0000 7f2978d63640 1 -- start start 2024-05-27T00:40:05.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2978d63640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29740af970 0x7f297415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2978d63640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29740b3a20 con 0x7f2974162430 2024-05-27T00:40:05.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2978d63640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29740b3d00 con 0x7f2974162050 2024-05-27T00:40:05.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2978d63640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f29740b4630 con 0x7f29740af970 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2972d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2974162430 0x7f29740b4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41856/0 (socket says 172.21.15.111:41856) 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2972d9d640 1 -- 172.21.15.111:0/1918665598 learned_addr learned my addr 172.21.15.111:0/1918665598 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108858564 0 0) 0x7f29740b3a20 con 0x7f2974162430 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2960003520 con 0x7f2974162430 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f296bfff640 1 --2- 172.21.15.111:0/1918665598 >> v2:172.21.15.153:3300/0 conn(0x7f29740af970 0x7f297415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3303111936 0 0) 0x7f2960003520 con 0x7f2974162430 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 >> v2:172.21.15.153:3300/0 conn(0x7f29740af970 msgr2=0x7f297415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 --2- 172.21.15.111:0/1918665598 >> v2:172.21.15.153:3300/0 conn(0x7f29740af970 0x7f297415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 >> v1:172.21.15.153:6789/0 conn(0x7f2974162050 legacy=0x7f29740af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29740b3fe0 con 0x7f2974162430 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f296c0019f0 con 0x7f2974162430 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 >> v1:172.21.15.111:6789/0 conn(0x7f2974162430 legacy=0x7f29740b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.509+0000 7f2971d9b640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2960003a00 0x7f2960003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2971d9b640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960004800 0x7f2960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2971d9b640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960005210 0x7f296000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f29740b4630 con 0x7f2960005210 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f296000c160 con 0x7f2960003a00 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f296000c360 con 0x7f2960004800 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2972d9d640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960005210 0x7f296000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f297259c640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2960003a00 0x7f2960003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f296bfff640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960004800 0x7f2960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2972d9d640 1 -- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960004800 msgr2=0x7f2960004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2972d9d640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960004800 0x7f2960004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2972d9d640 1 -- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2960003a00 msgr2=0x7f2960003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2972d9d640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2960003a00 0x7f2960003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f2972d9d640 1 -- 172.21.15.111:0/1918665598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2960003520 con 0x7f2960005210 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.510+0000 7f296bfff640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960004800 0x7f2960004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2972d9d640 1 -- 172.21.15.111:0/1918665598 reap_dead start 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2972d9d640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960005210 0x7f296000b9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f296c00a760 tx=0x7f2960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ae938ac1f829c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00a3b0 con 0x7f2960005210 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c02f070 con 0x7f2960005210 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2971d9b640 1 -- 172.21.15.111:0/1918665598 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c02bba0 con 0x7f2960005210 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 -- 172.21.15.111:0/1918665598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960005210 msgr2=0x7f296000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960005210 0x7f296000b9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f296c00a760 tx=0x7f2960005830 comp rx=0 tx=0).stop 2024-05-27T00:40:05.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 -- 172.21.15.111:0/1918665598 shutdown_connections 2024-05-27T00:40:05.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 --2- 172.21.15.111:0/1918665598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2960005210 0x7f296000b9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 -- 172.21.15.111:0/1918665598 >> 172.21.15.111:0/1918665598 conn(0x7f2974072a30 msgr2=0x7f297406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:05.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 -- 172.21.15.111:0/1918665598 shutdown_connections 2024-05-27T00:40:05.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.511+0000 7f2978d63640 1 -- 172.21.15.111:0/1918665598 wait complete. 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 Processor -- start 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 -- start start 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960005210 0x7f29741e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f297414e7a0 0x7f297406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 0x7f29741538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f297415fb30 con 0x7f2974140f70 2024-05-27T00:40:05.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f297415f9b0 con 0x7f297414e7a0 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2978d63640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f297415fcb0 con 0x7f2960005210 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.513+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 0x7f29741538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 0x7f29741538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42266/0 (socket says 172.21.15.111:42266) 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 -- 172.21.15.111:0/3828623167 learned_addr learned my addr 172.21.15.111:0/3828623167 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f296bfff640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f297414e7a0 0x7f297406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f297259c640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960005210 0x7f29741e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 -- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960005210 msgr2=0x7f29741e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960005210 0x7f29741e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 -- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f297414e7a0 msgr2=0x7f297406ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f297414e7a0 0x7f297406ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.521 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 -- 172.21.15.111:0/3828623167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29741e77c0 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2972d9d640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 0x7f29741538e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f296c00a880 tx=0x7f296c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c02bc30 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c02bdd0 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741e7a60 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0346b0 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.514+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741e8020 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.515+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974068680 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.516+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f296c034850 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.516+0000 7f296affd640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f295406be30 0x7f295406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.516+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f296c0ab2d0 con 0x7f2974140f70 2024-05-27T00:40:05.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.517+0000 7f297259c640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f295406be30 0x7f295406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.517+0000 7f297259c640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f295406be30 0x7f295406e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2964001760 tx=0x7f2964009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.521+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f296c046060 con 0x7f2974140f70 2024-05-27T00:40:05.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.697+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2974161450 con 0x7f2974140f70 2024-05-27T00:40:05.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.697+0000 7f296affd640 1 -- 172.21.15.111:0/3828623167 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f296c079210 con 0x7f2974140f70 2024-05-27T00:40:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.699+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f295406be30 msgr2=0x7f295406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f295406be30 0x7f295406e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2964001760 tx=0x7f2964009040 comp rx=0 tx=0).stop 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 msgr2=0x7f29741538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 0x7f29741538e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f296c00a880 tx=0x7f296c02d070 comp rx=0 tx=0).stop 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 shutdown_connections 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f295406be30 0x7f295406e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974140f70 0x7f29741538e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f297414e7a0 0x7f297406ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 --2- 172.21.15.111:0/3828623167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2960005210 0x7f29741e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 >> 172.21.15.111:0/3828623167 conn(0x7f2974072a30 msgr2=0x7f2974156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 shutdown_connections 2024-05-27T00:40:05.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.700+0000 7f2978d63640 1 -- 172.21.15.111:0/3828623167 wait complete. 2024-05-27T00:40:05.711 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:05.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.813+0000 7f1ca1831640 1 Processor -- start 2024-05-27T00:40:05.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.813+0000 7f1ca1831640 1 -- start start 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.814+0000 7f1ca1831640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c9c144000 0x7f1c9c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.814+0000 7f1ca1831640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1c9c06b390 0x7f1c9c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.814+0000 7f1ca1831640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9c0b3440 con 0x7f1c9c0cdc80 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.814+0000 7f1ca1831640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c0b42f0 con 0x7f1c9c06b390 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.814+0000 7f1ca1831640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c0b36e0 con 0x7f1c9c144000 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.814+0000 7f1c9a7fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1c9c06b390 0x7f1c9c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9b7fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1c9c0cdc80 0x7f1c9c0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41872/0 (socket says 172.21.15.111:41872) 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9b7fe640 1 -- 172.21.15.111:0/98506479 learned_addr learned my addr 172.21.15.111:0/98506479 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9affd640 1 --2- 172.21.15.111:0/98506479 >> v2:172.21.15.153:3300/0 conn(0x7f1c9c144000 0x7f1c9c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9a7fc640 1 -- 172.21.15.111:0/98506479 >> v2:172.21.15.153:3300/0 conn(0x7f1c9c144000 msgr2=0x7f1c9c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9a7fc640 1 --2- 172.21.15.111:0/98506479 >> v2:172.21.15.153:3300/0 conn(0x7f1c9c144000 0x7f1c9c0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9a7fc640 1 -- 172.21.15.111:0/98506479 >> v1:172.21.15.111:6789/0 conn(0x7f1c9c0cdc80 legacy=0x7f1c9c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c9a7fc640 1 -- 172.21.15.111:0/98506479 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c0b39c0 con 0x7f1c9c06b390 2024-05-27T00:40:05.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.815+0000 7f1c99ffb640 1 -- 172.21.15.111:0/98506479 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863712622 0 0) 0x7f1c9c0b3440 con 0x7f1c9c0cdc80 2024-05-27T00:40:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1c9a7fc640 1 --2- 172.21.15.111:0/98506479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c06b390 0x7f1c9c0cd740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0726a0 tx=0x7f1c9002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68c0fbc508c785e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1c99ffb640 1 -- 172.21.15.111:0/98506479 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90032770 con 0x7f1c9c06b390 2024-05-27T00:40:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1c99ffb640 1 -- 172.21.15.111:0/98506479 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c90038070 con 0x7f1c9c06b390 2024-05-27T00:40:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1c99ffb640 1 -- 172.21.15.111:0/98506479 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90032b30 con 0x7f1c9c06b390 2024-05-27T00:40:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 -- 172.21.15.111:0/98506479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c06b390 msgr2=0x7f1c9c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 --2- 172.21.15.111:0/98506479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c06b390 0x7f1c9c0cd740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0726a0 tx=0x7f1c9002c870 comp rx=0 tx=0).stop 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 -- 172.21.15.111:0/98506479 shutdown_connections 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 --2- 172.21.15.111:0/98506479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c06b390 0x7f1c9c0cd740 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 --2- 172.21.15.111:0/98506479 >> v2:172.21.15.153:3300/0 conn(0x7f1c9c144000 0x7f1c9c0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 -- 172.21.15.111:0/98506479 >> 172.21.15.111:0/98506479 conn(0x7f1c9c0b1c00 msgr2=0x7f1c9c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 -- 172.21.15.111:0/98506479 shutdown_connections 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.816+0000 7f1ca1831640 1 -- 172.21.15.111:0/98506479 wait complete. 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.817+0000 7f1ca1831640 1 Processor -- start 2024-05-27T00:40:05.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 -- start start 2024-05-27T00:40:05.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c9c06b390 0x7f1c9c1ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c9c0cdc80 0x7f1c9c0ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 0x7f1c9c1f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c072de0 con 0x7f1c9c144000 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c0b3f80 con 0x7f1c9c0cdc80 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.818+0000 7f1ca1831640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c072f60 con 0x7f1c9c06b390 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 0x7f1c9c1f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 0x7f1c9c1f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42284/0 (socket says 172.21.15.111:42284) 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 -- 172.21.15.111:0/4252498849 learned_addr learned my addr 172.21.15.111:0/4252498849 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9a7fc640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c9c0cdc80 0x7f1c9c0ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9affd640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c9c06b390 0x7f1c9c1ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 -- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c9c06b390 msgr2=0x7f1c9c1ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c9c06b390 0x7f1c9c1ef850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 -- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c9c0cdc80 msgr2=0x7f1c9c0ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c9c0cdc80 0x7f1c9c0ab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:05.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 -- 172.21.15.111:0/4252498849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c0ac750 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c9b7fe640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 0x7f1c9c1f2e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c480 tx=0x7f1c9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94018070 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94007500 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c0aca30 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94004a20 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.819+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c0acff0 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.820+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c068680 con 0x7f1c9c144000 2024-05-27T00:40:05.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.821+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c9401b070 con 0x7f1c9c144000 2024-05-27T00:40:05.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.821+0000 7f1c83fff640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1c8806be30 0x7f1c8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:05.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.821+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f1c9408a560 con 0x7f1c9c144000 2024-05-27T00:40:05.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.821+0000 7f1c9affd640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1c8806be30 0x7f1c8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:05.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.822+0000 7f1c9affd640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1c8806be30 0x7f1c8806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009fd0 tx=0x7f1c8c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:05.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:05.825+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c94058b80 con 0x7f1c9c144000 2024-05-27T00:40:06.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.000+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c9c073630 con 0x7f1c9c144000 2024-05-27T00:40:06.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.000+0000 7f1c83fff640 1 -- 172.21.15.111:0/4252498849 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f1c94058480 con 0x7f1c9c144000 2024-05-27T00:40:06.002 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:06.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1c8806be30 msgr2=0x7f1c8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1c8806be30 0x7f1c8806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009fd0 tx=0x7f1c8c00b040 comp rx=0 tx=0).stop 2024-05-27T00:40:06.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 msgr2=0x7f1c9c1f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 0x7f1c9c1f2e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c480 tx=0x7f1c9400c940 comp rx=0 tx=0).stop 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 shutdown_connections 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1c8806be30 0x7f1c8806e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c9c144000 0x7f1c9c1f2e50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c9c0cdc80 0x7f1c9c0ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 --2- 172.21.15.111:0/4252498849 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1c9c06b390 0x7f1c9c1ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 >> 172.21.15.111:0/4252498849 conn(0x7f1c9c0b1c00 msgr2=0x7f1c9c142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 shutdown_connections 2024-05-27T00:40:06.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.002+0000 7f1ca1831640 1 -- 172.21.15.111:0/4252498849 wait complete. 2024-05-27T00:40:06.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:06.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.116+0000 7f44b1fb6640 1 Processor -- start 2024-05-27T00:40:06.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.116+0000 7f44b1fb6640 1 -- start start 2024-05-27T00:40:06.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44b1fb6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac144000 0x7f44ac0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.118 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44b1fb6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f44ac06b390 0x7f44ac0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44b1fb6640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44ac0cdc80 0x7f44ac0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44b1fb6640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac0b42f0 con 0x7f44ac06b390 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44b1fb6640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac0b3440 con 0x7f44ac0cdc80 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44b1fb6640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0b36e0 con 0x7f44ac144000 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f44ac06b390 0x7f44ac0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f44ac06b390 0x7f44ac0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42288/0 (socket says 172.21.15.111:42288) 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 -- 172.21.15.111:0/3458496929 learned_addr learned my addr 172.21.15.111:0/3458496929 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44abfff640 1 --2- 172.21.15.111:0/3458496929 >> v2:172.21.15.153:3300/0 conn(0x7f44ac0cdc80 0x7f44ac0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44ab7fe640 1 --2- 172.21.15.111:0/3458496929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac144000 0x7f44ac0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 -- 172.21.15.111:0/3458496929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac144000 msgr2=0x7f44ac0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 --2- 172.21.15.111:0/3458496929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac144000 0x7f44ac0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 -- 172.21.15.111:0/3458496929 >> v2:172.21.15.153:3300/0 conn(0x7f44ac0cdc80 msgr2=0x7f44ac0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 --2- 172.21.15.111:0/3458496929 >> v2:172.21.15.153:3300/0 conn(0x7f44ac0cdc80 0x7f44ac0726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.117+0000 7f44aaffd640 1 -- 172.21.15.111:0/3458496929 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac0b39c0 con 0x7f44ac06b390 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.118+0000 7f44aaffd640 1 --2- 172.21.15.111:0/3458496929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac06b390 0x7f44ac0cd740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06cde0 tx=0x7f44a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4e01a23766787d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.118+0000 7f44aa7fc640 1 -- 172.21.15.111:0/3458496929 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0037070 con 0x7f44ac06b390 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.118+0000 7f44aa7fc640 1 -- 172.21.15.111:0/3458496929 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0032440 con 0x7f44ac06b390 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.118+0000 7f44aa7fc640 1 -- 172.21.15.111:0/3458496929 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0032810 con 0x7f44ac06b390 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 -- 172.21.15.111:0/3458496929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac06b390 msgr2=0x7f44ac0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/3458496929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac06b390 0x7f44ac0cd740 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06cde0 tx=0x7f44a002a1e0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 -- 172.21.15.111:0/3458496929 shutdown_connections 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/3458496929 >> v2:172.21.15.153:3300/0 conn(0x7f44ac0cdc80 0x7f44ac0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/3458496929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac06b390 0x7f44ac0cd740 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.121 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/3458496929 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac144000 0x7f44ac0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 -- 172.21.15.111:0/3458496929 >> 172.21.15.111:0/3458496929 conn(0x7f44ac0b1c00 msgr2=0x7f44ac0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:06.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 -- 172.21.15.111:0/3458496929 shutdown_connections 2024-05-27T00:40:06.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.119+0000 7f44b1fb6640 1 -- 172.21.15.111:0/3458496929 wait complete. 2024-05-27T00:40:06.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.120+0000 7f44b1fb6640 1 Processor -- start 2024-05-27T00:40:06.122 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.120+0000 7f44b1fb6640 1 -- start start 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44b1fb6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac06b390 0x7f44ac151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44b1fb6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac0cdc80 0x7f44ac06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44b1fb6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 0x7f44ac06c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44b1fb6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac072c10 con 0x7f44ac144000 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44b1fb6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0b3f80 con 0x7f44ac06b390 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44b1fb6640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac072d90 con 0x7f44ac0cdc80 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 0x7f44ac06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 0x7f44ac06c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42292/0 (socket says 172.21.15.111:42292) 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 -- 172.21.15.111:0/2845752278 learned_addr learned my addr 172.21.15.111:0/2845752278 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44ab7fe640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac06b390 0x7f44ac151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44aaffd640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac0cdc80 0x7f44ac06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 -- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac0cdc80 msgr2=0x7f44ac06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac0cdc80 0x7f44ac06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 -- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac06b390 msgr2=0x7f44ac151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac06b390 0x7f44ac151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44abfff640 1 -- 172.21.15.111:0/2845752278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac150f30 con 0x7f44ac144000 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.121+0000 7f44aaffd640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac0cdc80 0x7f44ac06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.122+0000 7f44abfff640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 0x7f44ac06c160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f44a400d5c0 tx=0x7f44a400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.122+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a40125f0 con 0x7f44ac144000 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.122+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a4019070 con 0x7f44ac144000 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.122+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac151210 con 0x7f44ac144000 2024-05-27T00:40:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.122+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a40129f0 con 0x7f44ac144000 2024-05-27T00:40:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.122+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac1517d0 con 0x7f44ac144000 2024-05-27T00:40:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.123+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac068680 con 0x7f44ac144000 2024-05-27T00:40:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.123+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44a4012b90 con 0x7f44ac144000 2024-05-27T00:40:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.124+0000 7f4493fff640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f449806be30 0x7f449806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.124+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f44a408b590 con 0x7f44ac144000 2024-05-27T00:40:06.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.124+0000 7f44ab7fe640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f449806be30 0x7f449806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.125+0000 7f44ab7fe640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f449806be30 0x7f449806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f449c0099c0 tx=0x7f449c003990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.128+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f44a4059bb0 con 0x7f44ac144000 2024-05-27T00:40:06.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.303+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44ac0afcc0 con 0x7f44ac144000 2024-05-27T00:40:06.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.303+0000 7f4493fff640 1 -- 172.21.15.111:0/2845752278 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f44a40594b0 con 0x7f44ac144000 2024-05-27T00:40:06.304 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f449806be30 msgr2=0x7f449806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f449806be30 0x7f449806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f449c0099c0 tx=0x7f449c003990 comp rx=0 tx=0).stop 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 msgr2=0x7f44ac06c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 0x7f44ac06c160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f44a400d5c0 tx=0x7f44a400da80 comp rx=0 tx=0).stop 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 shutdown_connections 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f449806be30 0x7f449806e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44ac144000 0x7f44ac06c160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f44ac0cdc80 0x7f44ac06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 --2- 172.21.15.111:0/2845752278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac06b390 0x7f44ac151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 >> 172.21.15.111:0/2845752278 conn(0x7f44ac0b1c00 msgr2=0x7f44ac0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:06.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 shutdown_connections 2024-05-27T00:40:06.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.305+0000 7f44b1fb6640 1 -- 172.21.15.111:0/2845752278 wait complete. 2024-05-27T00:40:06.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:06.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 Processor -- start 2024-05-27T00:40:06.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 -- start start 2024-05-27T00:40:06.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9bc0b3630 0x7fd9bc0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9bc06cc70 con 0x7fd9bc162390 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc0d1e50 con 0x7fd9bc0b3630 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.418+0000 7fd9c2b5a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0d2130 con 0x7fd9bc0b77e0 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c08f6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9bc0b3630 0x7fd9bc0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c10f7640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd9bc162390 0x7fd9bc0d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41886/0 (socket says 172.21.15.111:41886) 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c10f7640 1 -- 172.21.15.111:0/235155122 learned_addr learned my addr 172.21.15.111:0/235155122 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c08f6640 1 -- 172.21.15.111:0/235155122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 msgr2=0x7fd9bc161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c08f6640 1 --2- 172.21.15.111:0/235155122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c08f6640 1 -- 172.21.15.111:0/235155122 >> v1:172.21.15.111:6789/0 conn(0x7fd9bc162390 legacy=0x7fd9bc0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9c08f6640 1 -- 172.21.15.111:0/235155122 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc0d2410 con 0x7fd9bc0b3630 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.419+0000 7fd9bb7fe640 1 -- 172.21.15.111:0/235155122 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106719246 0 0) 0x7fd9bc06cc70 con 0x7fd9bc162390 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c08f6640 1 --2- 172.21.15.111:0/235155122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0b72a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40097a0 tx=0x7fd9b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a88013bc3f47966 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9bb7fe640 1 -- 172.21.15.111:0/235155122 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4037070 con 0x7fd9bc0b3630 2024-05-27T00:40:06.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9bb7fe640 1 -- 172.21.15.111:0/235155122 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b4032440 con 0x7fd9bc0b3630 2024-05-27T00:40:06.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9bb7fe640 1 -- 172.21.15.111:0/235155122 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4032810 con 0x7fd9bc0b3630 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/235155122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 msgr2=0x7fd9bc0b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/235155122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0b72a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40097a0 tx=0x7fd9b402a180 comp rx=0 tx=0).stop 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/235155122 shutdown_connections 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/235155122 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/235155122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0b72a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/235155122 >> 172.21.15.111:0/235155122 conn(0x7fd9bc0d42c0 msgr2=0x7fd9bc0d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/235155122 shutdown_connections 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.420+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/235155122 wait complete. 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.421+0000 7fd9c2b5a640 1 Processor -- start 2024-05-27T00:40:06.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 -- start start 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9bc162390 0x7fd9bc0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0d34a0 con 0x7fd9bc0b3630 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0d3320 con 0x7fd9bc162390 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c2b5a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0d3620 con 0x7fd9bc0b77e0 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.422+0000 7fd9c08f6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42298/0 (socket says 172.21.15.111:42298) 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 -- 172.21.15.111:0/4228431876 learned_addr learned my addr 172.21.15.111:0/4228431876 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c10f7640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9bc162390 0x7fd9bc0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 -- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 msgr2=0x7fd9bc1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9bbfff640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 -- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9bc162390 msgr2=0x7fd9bc0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9bc162390 0x7fd9bc0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 -- 172.21.15.111:0/4228431876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc159f30 con 0x7fd9bc0b3630 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c10f7640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9bc162390 0x7fd9bc0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9bbfff640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c08f6640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0cb1b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40098c0 tx=0x7fd9b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4041070 con 0x7fd9bc0b3630 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b403c870 con 0x7fd9bc0b3630 2024-05-27T00:40:06.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc0ce120 con 0x7fd9bc0b3630 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4037040 con 0x7fd9bc0b3630 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.423+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc0cbb90 con 0x7fd9bc0b3630 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.424+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc068680 con 0x7fd9bc0b3630 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.425+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9b4044070 con 0x7fd9bc0b3630 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.425+0000 7fd9b97fa640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd99c06be30 0x7fd99c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.425+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fd9b40abd30 con 0x7fd9bc0b3630 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.425+0000 7fd9bbfff640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd99c06be30 0x7fd99c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.426+0000 7fd9bbfff640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd99c06be30 0x7fd99c06e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0b7380 tx=0x7fd9ac0073c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.430+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd9b407a470 con 0x7fd9bc0b3630 2024-05-27T00:40:06.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.602+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd9bc0ce470 con 0x7fd9bc0b3630 2024-05-27T00:40:06.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.603+0000 7fd9b97fa640 1 -- 172.21.15.111:0/4228431876 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fd9b4079d70 con 0x7fd9bc0b3630 2024-05-27T00:40:06.604 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd99c06be30 msgr2=0x7fd99c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd99c06be30 0x7fd99c06e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0b7380 tx=0x7fd9ac0073c0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 msgr2=0x7fd9bc0cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0cb1b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40098c0 tx=0x7fd9b40046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 shutdown_connections 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd99c06be30 0x7fd99c06e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9bc162390 0x7fd9bc0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd9bc0b77e0 0x7fd9bc1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 --2- 172.21.15.111:0/4228431876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9bc0b3630 0x7fd9bc0cb1b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 >> 172.21.15.111:0/4228431876 conn(0x7fd9bc0d42c0 msgr2=0x7fd9bc0d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:06.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 shutdown_connections 2024-05-27T00:40:06.607 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.605+0000 7fd9c2b5a640 1 -- 172.21.15.111:0/4228431876 wait complete. 2024-05-27T00:40:06.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-05-27T00:40:06.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.715+0000 7fdab524d640 1 Processor -- start 2024-05-27T00:40:06.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.716+0000 7fdab524d640 1 -- start start 2024-05-27T00:40:06.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.716+0000 7fdab524d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013f340 0x7fdab00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.716+0000 7fdab524d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdab00cce00 0x7fdab0141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.716+0000 7fdab524d640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab0142650 con 0x7fdab013ef60 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.716+0000 7fdab524d640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdab00cf210 con 0x7fdab00cce00 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.716+0000 7fdab524d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdab00cebc0 con 0x7fdab013f340 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaa7fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013f340 0x7fdab00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaaf59e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdab00cce00 0x7fdab0141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaa7fff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013f340 0x7fdab00cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:57116/0 (socket says 172.21.15.111:57116) 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaa7fff640 1 -- 172.21.15.111:0/1192866529 learned_addr learned my addr 172.21.15.111:0/1192866529 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaaed9d640 1 --1- 172.21.15.111:0/1192866529 >> v1:172.21.15.153:6789/0 conn(0x7fdab013ef60 0x7fdab00cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:42284/0 (socket says 172.21.15.111:42284) 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaaf59e640 1 -- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013f340 msgr2=0x7fdab00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:06.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaaf59e640 1 --2- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013f340 0x7fdab00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaaf59e640 1 -- 172.21.15.111:0/1192866529 >> v1:172.21.15.153:6789/0 conn(0x7fdab013ef60 legacy=0x7fdab00cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.717+0000 7fdaaf59e640 1 -- 172.21.15.111:0/1192866529 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab00ceea0 con 0x7fdab00cce00 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdaaf59e640 1 --2- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0141920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009d90 tx=0x7fdaa802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f13b3ff70e4a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdaae59c640 1 -- 172.21.15.111:0/1192866529 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8007740 con 0x7fdab00cce00 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdaae59c640 1 -- 172.21.15.111:0/1192866529 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa80078e0 con 0x7fdab00cce00 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdaae59c640 1 -- 172.21.15.111:0/1192866529 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8007cb0 con 0x7fdab00cce00 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdab524d640 1 -- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 msgr2=0x7fdab0141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdab524d640 1 --2- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0141920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009d90 tx=0x7fdaa802a660 comp rx=0 tx=0).stop 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdab524d640 1 -- 172.21.15.111:0/1192866529 shutdown_connections 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdab524d640 1 --2- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0141920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdab524d640 1 --2- 172.21.15.111:0/1192866529 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013f340 0x7fdab00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.718+0000 7fdab524d640 1 -- 172.21.15.111:0/1192866529 >> 172.21.15.111:0/1192866529 conn(0x7fdab00b1c00 msgr2=0x7fdab00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:06.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.719+0000 7fdab524d640 1 -- 172.21.15.111:0/1192866529 shutdown_connections 2024-05-27T00:40:06.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.719+0000 7fdab524d640 1 -- 172.21.15.111:0/1192866529 wait complete. 2024-05-27T00:40:06.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 Processor -- start 2024-05-27T00:40:06.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 -- start start 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013ef60 0x7fdab01f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 0x7fdab01409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0141fe0 con 0x7fdab013f340 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0141e60 con 0x7fdab00cce00 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.720+0000 7fdab524d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdab0142160 con 0x7fdab013ef60 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 0x7fdab01409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 0x7fdab01409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42300/0 (socket says 172.21.15.111:42300) 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 -- 172.21.15.111:0/2534606336 learned_addr learned my addr 172.21.15.111:0/2534606336 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaed9d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaa7fff640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013ef60 0x7fdab01f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 -- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013ef60 msgr2=0x7fdab01f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013ef60 0x7fdab01f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 -- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 msgr2=0x7fdab0158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 -- 172.21.15.111:0/2534606336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab01f2080 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaaf59e640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 0x7fdab01409d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009eb0 tx=0x7fdaa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8040070 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa8031bc0 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01f2360 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8009450 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.721+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01f2920 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.722+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab0068680 con 0x7fdab013f340 2024-05-27T00:40:06.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.723+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdaa8039040 con 0x7fdab013f340 2024-05-27T00:40:06.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.723+0000 7fdaa77fe640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fda9006be30 0x7fda9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:06.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.724+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fdaa80ac9c0 con 0x7fdab013f340 2024-05-27T00:40:06.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.724+0000 7fdaaed9d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fda9006be30 0x7fda9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:06.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.724+0000 7fdaaed9d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fda9006be30 0x7fda9006e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007fd0 tx=0x7fdaa0007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:06.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.728+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdaa80aee20 con 0x7fdab013f340 2024-05-27T00:40:06.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:06.967+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7fdab01592c0 con 0x7fdab013f340 2024-05-27T00:40:07.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.061+0000 7fdaa77fe640 1 -- 172.21.15.111:0/2534606336 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v64) ==== 147+0+0 (secure 0 0 0) 0x7fdaa807a900 con 0x7fdab013f340 2024-05-27T00:40:07.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fda9006be30 msgr2=0x7fda9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fda9006be30 0x7fda9006e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007fd0 tx=0x7fdaa0007e80 comp rx=0 tx=0).stop 2024-05-27T00:40:07.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 msgr2=0x7fdab01409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 0x7fdab01409d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009eb0 tx=0x7fdaa8004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:07.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 shutdown_connections 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fda9006be30 0x7fda9006e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdab013f340 0x7fdab01409d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdab013ef60 0x7fdab01f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 --2- 172.21.15.111:0/2534606336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab00cce00 0x7fdab0158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 >> 172.21.15.111:0/2534606336 conn(0x7fdab00b1c00 msgr2=0x7fdab0160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 shutdown_connections 2024-05-27T00:40:07.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.063+0000 7fdab524d640 1 -- 172.21.15.111:0/2534606336 wait complete. 2024-05-27T00:40:07.075 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-27T00:40:07.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.177+0000 7f522d1ee640 1 Processor -- start 2024-05-27T00:40:07.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.177+0000 7f522d1ee640 1 -- start start 2024-05-27T00:40:07.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522d1ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5228144000 0x7f52280b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522d1ee640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52280726d0 0x7f5228072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522d1ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5228073020 con 0x7f522806b390 2024-05-27T00:40:07.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522d1ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f52280b42f0 con 0x7f5228144000 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522d1ee640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f52280b3440 con 0x7f52280726d0 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f5226d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5228144000 0x7f52280b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522659c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f522806b390 0x7f52280cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41898/0 (socket says 172.21.15.111:41898) 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.178+0000 7f522659c640 1 -- 172.21.15.111:0/592516882 learned_addr learned my addr 172.21.15.111:0/592516882 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f5226d9d640 1 -- 172.21.15.111:0/592516882 >> v2:172.21.15.153:3300/0 conn(0x7f52280726d0 msgr2=0x7f5228072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f522759e640 1 --2- 172.21.15.111:0/592516882 >> v2:172.21.15.153:3300/0 conn(0x7f52280726d0 0x7f5228072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f5226d9d640 1 --2- 172.21.15.111:0/592516882 >> v2:172.21.15.153:3300/0 conn(0x7f52280726d0 0x7f5228072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f5225d9b640 1 -- 172.21.15.111:0/592516882 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596283170 0 0) 0x7f521c001110 con 0x7f522806b390 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f5226d9d640 1 -- 172.21.15.111:0/592516882 >> v1:172.21.15.111:6789/0 conn(0x7f522806b390 legacy=0x7f52280cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f5226d9d640 1 -- 172.21.15.111:0/592516882 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52280b36e0 con 0x7f5228144000 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.179+0000 7f5226d9d640 1 --2- 172.21.15.111:0/592516882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5228144000 0x7f52280b2f00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f52280b2ca0 tx=0x7f521802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44d24eb9834fb4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f5225d9b640 1 -- 172.21.15.111:0/592516882 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218007740 con 0x7f5228144000 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f5225d9b640 1 -- 172.21.15.111:0/592516882 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52180078e0 con 0x7f5228144000 2024-05-27T00:40:07.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f5225d9b640 1 -- 172.21.15.111:0/592516882 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218007cb0 con 0x7f5228144000 2024-05-27T00:40:07.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 -- 172.21.15.111:0/592516882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5228144000 msgr2=0x7f52280b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 --2- 172.21.15.111:0/592516882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5228144000 0x7f52280b2f00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f52280b2ca0 tx=0x7f521802a680 comp rx=0 tx=0).stop 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 -- 172.21.15.111:0/592516882 shutdown_connections 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 --2- 172.21.15.111:0/592516882 >> v2:172.21.15.153:3300/0 conn(0x7f52280726d0 0x7f5228072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 --2- 172.21.15.111:0/592516882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5228144000 0x7f52280b2f00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 -- 172.21.15.111:0/592516882 >> 172.21.15.111:0/592516882 conn(0x7f52280b1c00 msgr2=0x7f52280b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 -- 172.21.15.111:0/592516882 shutdown_connections 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.180+0000 7f522d1ee640 1 -- 172.21.15.111:0/592516882 wait complete. 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 Processor -- start 2024-05-27T00:40:07.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 -- start start 2024-05-27T00:40:07.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f522806b390 0x7f52280ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 0x7f5228159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5228144000 0x7f5228165f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f52280b3f80 con 0x7f52280726d0 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f52280b3ca0 con 0x7f5228144000 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522d1ee640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f52280cde50 con 0x7f522806b390 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 0x7f5228159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.182+0000 7f522659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 0x7f5228159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42322/0 (socket says 172.21.15.111:42322) 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 -- 172.21.15.111:0/2186988417 learned_addr learned my addr 172.21.15.111:0/2186988417 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:07.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522759e640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5228144000 0x7f5228165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 -- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f522806b390 msgr2=0x7f52280ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f522806b390 0x7f52280ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 -- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5228144000 msgr2=0x7f5228165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5228144000 0x7f5228165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 -- 172.21.15.111:0/2186988417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52280ade80 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522759e640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5228144000 0x7f5228165f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522659c640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 0x7f5228159f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c410 tx=0x7f521c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0074e0 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c0047e0 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52280ae160 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c004bd0 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.183+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52280ae720 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.184+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5228068680 con 0x7f52280726d0 2024-05-27T00:40:07.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.184+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f521c011070 con 0x7f52280726d0 2024-05-27T00:40:07.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.185+0000 7f520f7fe640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f521406be30 0x7f521406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.185+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f521c08ae80 con 0x7f52280726d0 2024-05-27T00:40:07.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.185+0000 7f5226d9d640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f521406be30 0x7f521406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.186+0000 7f5226d9d640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f521406be30 0x7f521406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5218034000 tx=0x7f521802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:07.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.189+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5228073020 con 0x7f52280726d0 2024-05-27T00:40:07.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.364+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f522815a440 con 0x7f52280726d0 2024-05-27T00:40:07.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.367+0000 7f520f7fe640 1 -- 172.21.15.111:0/2186988417 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f521c0594a0 con 0x7f52280726d0 2024-05-27T00:40:07.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f521406be30 msgr2=0x7f521406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f521406be30 0x7f521406e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5218034000 tx=0x7f521802e000 comp rx=0 tx=0).stop 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 msgr2=0x7f5228159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 0x7f5228159f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c410 tx=0x7f521c00c8d0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 shutdown_connections 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f521406be30 0x7f521406e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5228144000 0x7f5228165f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52280726d0 0x7f5228159f00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 --2- 172.21.15.111:0/2186988417 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f522806b390 0x7f52280ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 >> 172.21.15.111:0/2186988417 conn(0x7f52280b1c00 msgr2=0x7f5228143130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 shutdown_connections 2024-05-27T00:40:07.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.369+0000 7f522d1ee640 1 -- 172.21.15.111:0/2186988417 wait complete. 2024-05-27T00:40:07.382 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-27T00:40:07.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-27T00:40:07.427 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-27T00:40:07.480 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-27T00:40:07.480 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:07.480 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:07.495 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:07.496 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:07.552 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:07.552 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:07.607 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:07.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.746+0000 7f0e8769e640 1 Processor -- start 2024-05-27T00:40:07.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.746+0000 7f0e8769e640 1 -- start start 2024-05-27T00:40:07.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8769e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 0x7f0e8015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8769e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8769e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e80160b30 con 0x7f0e8006b390 2024-05-27T00:40:07.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8769e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0e800cec10 con 0x7f0e8015c250 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8769e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0e80160db0 con 0x7f0e800cdc10 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e85c3b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8543a640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 0x7f0e8015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8543a640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 0x7f0e8015fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:43024/0 (socket says 172.21.15.111:43024) 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e85c3b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:57142/0 (socket says 172.21.15.111:57142) 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e8543a640 1 -- 172.21.15.111:0/1101205096 learned_addr learned my addr 172.21.15.111:0/1101205096 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.747+0000 7f0e84c39640 1 --1- 172.21.15.111:0/1101205096 >> v1:172.21.15.153:6789/0 conn(0x7f0e8006b390 0x7f0e80160420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:42292/0 (socket says 172.21.15.111:42292) 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.748+0000 7f0e85c3b640 1 -- 172.21.15.111:0/1101205096 >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 msgr2=0x7f0e8015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.748+0000 7f0e85c3b640 1 --2- 172.21.15.111:0/1101205096 >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 0x7f0e8015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.748+0000 7f0e85c3b640 1 -- 172.21.15.111:0/1101205096 >> v1:172.21.15.153:6789/0 conn(0x7f0e8006b390 legacy=0x7f0e80160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.748+0000 7f0e85c3b640 1 -- 172.21.15.111:0/1101205096 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800b3e50 con 0x7f0e800cdc10 2024-05-27T00:40:07.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.748+0000 7f0e8543a640 1 --2- 172.21.15.111:0/1101205096 >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 0x7f0e8015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:07.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.748+0000 7f0e85c3b640 1 --2- 172.21.15.111:0/1101205096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800cf3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009d90 tx=0x7f0e7802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab497c241dfe6f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e77fff640 1 -- 172.21.15.111:0/1101205096 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78031600 con 0x7f0e800cdc10 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e77fff640 1 -- 172.21.15.111:0/1101205096 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780317a0 con 0x7f0e800cdc10 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e77fff640 1 -- 172.21.15.111:0/1101205096 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78031b70 con 0x7f0e800cdc10 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 -- 172.21.15.111:0/1101205096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 msgr2=0x7f0e800cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 --2- 172.21.15.111:0/1101205096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800cf3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009d90 tx=0x7f0e7802a5d0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 -- 172.21.15.111:0/1101205096 shutdown_connections 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 --2- 172.21.15.111:0/1101205096 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800cf3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 --2- 172.21.15.111:0/1101205096 >> v2:172.21.15.153:3300/0 conn(0x7f0e8015c250 0x7f0e8015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 -- 172.21.15.111:0/1101205096 >> 172.21.15.111:0/1101205096 conn(0x7f0e80071b20 msgr2=0x7f0e800b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 -- 172.21.15.111:0/1101205096 shutdown_connections 2024-05-27T00:40:07.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.749+0000 7f0e8769e640 1 -- 172.21.15.111:0/1101205096 wait complete. 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.750+0000 7f0e8769e640 1 Processor -- start 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.750+0000 7f0e8769e640 1 -- start start 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8769e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 0x7f0e8015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8769e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8769e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8015c250 0x7f0e800b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8769e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8015c630 con 0x7f0e8006b390 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8769e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800b4130 con 0x7f0e8015c250 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8769e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8015c7b0 con 0x7f0e800cdc10 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 0x7f0e8015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 0x7f0e8015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42328/0 (socket says 172.21.15.111:42328) 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 -- 172.21.15.111:0/3886600260 learned_addr learned my addr 172.21.15.111:0/3886600260 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e84c39640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e85c3b640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8015c250 0x7f0e800b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 -- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 msgr2=0x7f0e800b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 -- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8015c250 msgr2=0x7f0e800b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8015c250 0x7f0e800b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.751+0000 7f0e8543a640 1 -- 172.21.15.111:0/3886600260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e801532b0 con 0x7f0e8006b390 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e85c3b640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8015c250 0x7f0e800b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e8543a640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 0x7f0e8015f2d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00ea00 tx=0x7f0e7c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c017070 con 0x7f0e8006b390 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7c00ca60 con 0x7f0e8006b390 2024-05-27T00:40:07.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8015d160 con 0x7f0e8006b390 2024-05-27T00:40:07.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00ce10 con 0x7f0e8006b390 2024-05-27T00:40:07.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.752+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e80157930 con 0x7f0e8006b390 2024-05-27T00:40:07.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.753+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e80068680 con 0x7f0e8006b390 2024-05-27T00:40:07.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.754+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e7c01a070 con 0x7f0e8006b390 2024-05-27T00:40:07.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.754+0000 7f0e75ffb640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0e6406be30 0x7f0e6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:07.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.754+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f0e7c08b260 con 0x7f0e8006b390 2024-05-27T00:40:07.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.754+0000 7f0e84c39640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0e6406be30 0x7f0e6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:07.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.755+0000 7f0e84c39640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0e6406be30 0x7f0e6406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e70007fd0 tx=0x7f0e70007c40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:07.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.759+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0e7c0597f0 con 0x7f0e8006b390 2024-05-27T00:40:07.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.935+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e80075620 con 0x7f0e8006b390 2024-05-27T00:40:07.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.936+0000 7f0e75ffb640 1 -- 172.21.15.111:0/3886600260 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f0e7c0590f0 con 0x7f0e8006b390 2024-05-27T00:40:07.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:07.937 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0e6406be30 msgr2=0x7f0e6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0e6406be30 0x7f0e6406e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e70007fd0 tx=0x7f0e70007c40 comp rx=0 tx=0).stop 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 msgr2=0x7f0e8015f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 0x7f0e8015f2d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00ea00 tx=0x7f0e7c00eec0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 shutdown_connections 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0e6406be30 0x7f0e6406e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e8015c250 0x7f0e800b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0e800cdc10 0x7f0e800b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 --2- 172.21.15.111:0/3886600260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e8006b390 0x7f0e8015f2d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:07.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 >> 172.21.15.111:0/3886600260 conn(0x7f0e80071b20 msgr2=0x7f0e800cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:07.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 shutdown_connections 2024-05-27T00:40:07.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:07.938+0000 7f0e8769e640 1 -- 172.21.15.111:0/3886600260 wait complete. 2024-05-27T00:40:07.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:07.950 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:08.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.052+0000 7fef2a67b640 1 Processor -- start 2024-05-27T00:40:08.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.052+0000 7fef2a67b640 1 -- start start 2024-05-27T00:40:08.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef2a67b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef24071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef2a67b640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef24071fe0 con 0x7fef24071b40 2024-05-27T00:40:08.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef2a67b640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef240722c0 con 0x7fef2406c330 2024-05-27T00:40:08.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef2a67b640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fef240725a0 con 0x7fef24151740 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef1bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fef24071b40 0x7fef2406bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41914/0 (socket says 172.21.15.111:41914) 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef1bfff640 1 -- 172.21.15.111:0/3422325803 learned_addr learned my addr 172.21.15.111:0/3422325803 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825461020 0 0) 0x7fef24071fe0 con 0x7fef24071b40 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef10004630 con 0x7fef24071b40 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.053+0000 7fef23fff640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef24071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2658972408 0 0) 0x7fef10004630 con 0x7fef24071b40 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 msgr2=0x7fef24071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef24071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 >> v1:172.21.15.153:6789/0 conn(0x7fef2406c330 legacy=0x7fef240cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef24072880 con 0x7fef24071b40 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fef0c001670 con 0x7fef24071b40 2024-05-27T00:40:08.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef23fff640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef24071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 >> v1:172.21.15.111:6789/0 conn(0x7fef24071b40 legacy=0x7fef2406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef10004af0 0x7fef10004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef10005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef10005e50 0x7fef100062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef240725a0 con 0x7fef10005e50 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef100035f0 con 0x7fef10004af0 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fef100037f0 con 0x7fef24151740 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef28c18640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef10005e50 0x7fef100062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef23fff640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef10004af0 0x7fef10004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef1bfff640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef10005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef28c18640 1 -- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 msgr2=0x7fef10005910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:08.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef28c18640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef24151740 0x7fef10005910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef28c18640 1 -- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef10004af0 msgr2=0x7fef10004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef28c18640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef10004af0 0x7fef10004f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.054+0000 7fef28c18640 1 -- 172.21.15.111:0/3422325803 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef10004630 con 0x7fef10005e50 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.055+0000 7fef28c18640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef10005e50 0x7fef100062d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef240b2cc0 tx=0x7fef10006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966d1ea8d23dbdb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.055+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c001a70 con 0x7fef10005e50 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.055+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1c02ae10 con 0x7fef10005e50 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.055+0000 7fef237fe640 1 -- 172.21.15.111:0/3422325803 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c003bb0 con 0x7fef10005e50 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef2a67b640 1 -- 172.21.15.111:0/3422325803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef10005e50 msgr2=0x7fef100062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef2a67b640 1 --2- 172.21.15.111:0/3422325803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef10005e50 0x7fef100062d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef240b2cc0 tx=0x7fef10006940 comp rx=0 tx=0).stop 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef28c18640 1 -- 172.21.15.111:0/3422325803 reap_dead start 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef2a67b640 1 -- 172.21.15.111:0/3422325803 shutdown_connections 2024-05-27T00:40:08.148 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef2a67b640 1 -- 172.21.15.111:0/3422325803 >> 172.21.15.111:0/3422325803 conn(0x7fef240b1c00 msgr2=0x7fef240b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef2a67b640 1 -- 172.21.15.111:0/3422325803 shutdown_connections 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.056+0000 7fef2a67b640 1 -- 172.21.15.111:0/3422325803 wait complete. 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.057+0000 7fef2a67b640 1 Processor -- start 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.057+0000 7fef2a67b640 1 -- start start 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.057+0000 7fef2a67b640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef10005e50 0x7fef240aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 0x7fef24159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef241eefc0 0x7fef241ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef2415b090 con 0x7fef241422a0 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef2415af10 con 0x7fef241eefc0 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fef240b42f0 con 0x7fef10005e50 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 0x7fef24159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 0x7fef24159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42336/0 (socket says 172.21.15.111:42336) 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 -- 172.21.15.111:0/560538966 learned_addr learned my addr 172.21.15.111:0/560538966 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef23fff640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef10005e50 0x7fef240aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef28c18640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef241eefc0 0x7fef241ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 -- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef10005e50 msgr2=0x7fef240aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef10005e50 0x7fef240aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 -- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef241eefc0 msgr2=0x7fef241ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef241eefc0 0x7fef241ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 -- 172.21.15.111:0/560538966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef24142720 con 0x7fef241422a0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef1bfff640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 0x7fef24159230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007340 tx=0x7fef0c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f940 con 0x7fef241422a0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c018070 con 0x7fef241422a0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00fd60 con 0x7fef241422a0 2024-05-27T00:40:08.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241ef9b0 con 0x7fef241422a0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.058+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef24142cc0 con 0x7fef241422a0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.059+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef24068680 con 0x7fef241422a0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.060+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef0c01d460 con 0x7fef241422a0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.060+0000 7fef217fa640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef0406be60 0x7fef0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.060+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fef0c08b440 con 0x7fef241422a0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.060+0000 7fef23fff640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef0406be60 0x7fef0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.061+0000 7fef23fff640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef0406be60 0x7fef0406e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef14006fd0 tx=0x7fef14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.151 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.064+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fef0c059940 con 0x7fef241422a0 2024-05-27T00:40:08.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.240+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef24073030 con 0x7fef241422a0 2024-05-27T00:40:08.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.241+0000 7fef217fa640 1 -- 172.21.15.111:0/560538966 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fef0c059240 con 0x7fef241422a0 2024-05-27T00:40:08.242 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef0406be60 msgr2=0x7fef0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef0406be60 0x7fef0406e320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef14006fd0 tx=0x7fef14008040 comp rx=0 tx=0).stop 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 msgr2=0x7fef24159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 0x7fef24159230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007340 tx=0x7fef0c007800 comp rx=0 tx=0).stop 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 shutdown_connections 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef0406be60 0x7fef0406e320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef241eefc0 0x7fef241ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef241422a0 0x7fef24159230 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 --2- 172.21.15.111:0/560538966 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef10005e50 0x7fef240aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 >> 172.21.15.111:0/560538966 conn(0x7fef240b1c00 msgr2=0x7fef240afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 shutdown_connections 2024-05-27T00:40:08.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.243+0000 7fef2a67b640 1 -- 172.21.15.111:0/560538966 wait complete. 2024-05-27T00:40:08.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:08.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.357+0000 7f08b60ee640 1 Processor -- start 2024-05-27T00:40:08.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.357+0000 7f08b60ee640 1 -- start start 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08b60ee640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08b0144000 0x7f08b00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08b60ee640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b00b3440 con 0x7f08b00726d0 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08b60ee640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b00b36e0 con 0x7f08b006b390 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08b60ee640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f08b00b42f0 con 0x7f08b0144000 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08af7fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08b0144000 0x7f08b00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08affff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f08b00726d0 0x7f08b0072ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41930/0 (socket says 172.21.15.111:41930) 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08affff640 1 -- 172.21.15.111:0/2865515400 learned_addr learned my addr 172.21.15.111:0/2865515400 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:08.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08af7fe640 1 -- 172.21.15.111:0/2865515400 >> v1:172.21.15.153:6789/0 conn(0x7f08b006b390 legacy=0x7f08b00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:08.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08af7fe640 1 -- 172.21.15.111:0/2865515400 >> v1:172.21.15.111:6789/0 conn(0x7f08b00726d0 legacy=0x7f08b0072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08af7fe640 1 -- 172.21.15.111:0/2865515400 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b00b39c0 con 0x7f08b0144000 2024-05-27T00:40:08.360 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.358+0000 7f08ae7fc640 1 -- 172.21.15.111:0/2865515400 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290181831 0 0) 0x7f08b00b3440 con 0x7f08b00726d0 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08af7fe640 1 --2- 172.21.15.111:0/2865515400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b0144000 0x7f08b00b2f00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08b00b4470 tx=0x7f08a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe398fbaf13413f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08ae7fc640 1 -- 172.21.15.111:0/2865515400 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0037070 con 0x7f08b0144000 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08ae7fc640 1 -- 172.21.15.111:0/2865515400 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a0032440 con 0x7f08b0144000 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08ae7fc640 1 -- 172.21.15.111:0/2865515400 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0032810 con 0x7f08b0144000 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08b60ee640 1 -- 172.21.15.111:0/2865515400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b0144000 msgr2=0x7f08b00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08b60ee640 1 --2- 172.21.15.111:0/2865515400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b0144000 0x7f08b00b2f00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08b00b4470 tx=0x7f08a002a1c0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08b60ee640 1 -- 172.21.15.111:0/2865515400 shutdown_connections 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08b60ee640 1 --2- 172.21.15.111:0/2865515400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b0144000 0x7f08b00b2f00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.359+0000 7f08b60ee640 1 -- 172.21.15.111:0/2865515400 >> 172.21.15.111:0/2865515400 conn(0x7f08b00b1c00 msgr2=0x7f08b00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.360+0000 7f08b60ee640 1 -- 172.21.15.111:0/2865515400 shutdown_connections 2024-05-27T00:40:08.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.360+0000 7f08b60ee640 1 -- 172.21.15.111:0/2865515400 wait complete. 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.361+0000 7f08b60ee640 1 Processor -- start 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.361+0000 7f08b60ee640 1 -- start start 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08b006b390 0x7f08b013c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 0x7f08b00ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08b0144000 0x7f08b00acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00cde50 con 0x7f08b00726d0 2024-05-27T00:40:08.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00b3f80 con 0x7f08b0144000 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f08b00b4780 con 0x7f08b006b390 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 0x7f08b00ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 0x7f08b00ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42358/0 (socket says 172.21.15.111:42358) 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 -- 172.21.15.111:0/1155770519 learned_addr learned my addr 172.21.15.111:0/1155770519 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08af7fe640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08b006b390 0x7f08b013c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08affff640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08b0144000 0x7f08b00acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 -- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08b006b390 msgr2=0x7f08b013c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08b006b390 0x7f08b013c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 -- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08b0144000 msgr2=0x7f08b00acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08b0144000 0x7f08b00acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 -- 172.21.15.111:0/1155770519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b0141d70 con 0x7f08b00726d0 2024-05-27T00:40:08.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08affff640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08b0144000 0x7f08b00acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08af7fe640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08b006b390 0x7f08b013c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08aeffd640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 0x7f08b00ac590 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f089800c400 tx=0x7f089800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898011790 con 0x7f08b00726d0 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898017070 con 0x7f08b00726d0 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b0142050 con 0x7f08b00726d0 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898011bb0 con 0x7f08b00726d0 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.362+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b0142610 con 0x7f08b00726d0 2024-05-27T00:40:08.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.363+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b01429f0 con 0x7f08b00726d0 2024-05-27T00:40:08.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.364+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f089801c460 con 0x7f08b00726d0 2024-05-27T00:40:08.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.364+0000 7f08a7fff640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f089006be30 0x7f089006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.364+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f089808b0c0 con 0x7f08b00726d0 2024-05-27T00:40:08.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.365+0000 7f08af7fe640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f089006be30 0x7f089006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.365+0000 7f08af7fe640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f089006be30 0x7f089006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f08a0034000 tx=0x7f08a002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.369+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f08b01429f0 con 0x7f08b00726d0 2024-05-27T00:40:08.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.543+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08b0068680 con 0x7f08b00726d0 2024-05-27T00:40:08.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.543+0000 7f08a7fff640 1 -- 172.21.15.111:0/1155770519 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f08980596e0 con 0x7f08b00726d0 2024-05-27T00:40:08.545 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f089006be30 msgr2=0x7f089006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f089006be30 0x7f089006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f08a0034000 tx=0x7f08a002e000 comp rx=0 tx=0).stop 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 msgr2=0x7f08b00ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 0x7f08b00ac590 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f089800c400 tx=0x7f089800c8c0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 shutdown_connections 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f089006be30 0x7f089006e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08b0144000 0x7f08b00acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b00726d0 0x7f08b00ac590 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 --2- 172.21.15.111:0/1155770519 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f08b006b390 0x7f08b013c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.545+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 >> 172.21.15.111:0/1155770519 conn(0x7f08b00b1c00 msgr2=0x7f08b0143320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.546+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 shutdown_connections 2024-05-27T00:40:08.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.546+0000 7f08b60ee640 1 -- 172.21.15.111:0/1155770519 wait complete. 2024-05-27T00:40:08.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:08.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.659+0000 7f63fc561640 1 Processor -- start 2024-05-27T00:40:08.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.659+0000 7f63fc561640 1 -- start start 2024-05-27T00:40:08.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63fc561640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4151740 0x7f63f4071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63fc561640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f4071fe0 con 0x7f63f4071b40 2024-05-27T00:40:08.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63fc561640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f40722c0 con 0x7f63f406c330 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63fc561640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f63f40725a0 con 0x7f63f4151740 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63f9afc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f63f4071b40 0x7f63f406bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41936/0 (socket says 172.21.15.111:41936) 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63f9afc640 1 -- 172.21.15.111:0/747487998 learned_addr learned my addr 172.21.15.111:0/747487998 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63fa2fd640 1 --2- 172.21.15.111:0/747487998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4151740 0x7f63f4071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63f92fb640 1 -- 172.21.15.111:0/747487998 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722258496 0 0) 0x7f63f4071fe0 con 0x7f63f4071b40 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.660+0000 7f63f92fb640 1 -- 172.21.15.111:0/747487998 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63e0003520 con 0x7f63f4071b40 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.661+0000 7f63f92fb640 1 -- 172.21.15.111:0/747487998 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186471458 0 0) 0x7f63f40722c0 con 0x7f63f406c330 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.661+0000 7f63fa2fd640 1 -- 172.21.15.111:0/747487998 >> v1:172.21.15.153:6789/0 conn(0x7f63f406c330 legacy=0x7f63f40cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.661+0000 7f63fa2fd640 1 -- 172.21.15.111:0/747487998 >> v1:172.21.15.111:6789/0 conn(0x7f63f4071b40 legacy=0x7f63f406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.661+0000 7f63fa2fd640 1 -- 172.21.15.111:0/747487998 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f4072880 con 0x7f63f4151740 2024-05-27T00:40:08.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.661+0000 7f63fa2fd640 1 --2- 172.21.15.111:0/747487998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4151740 0x7f63f4071600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009dc0 tx=0x7f63f002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53655910f80a592 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63f92fb640 1 -- 172.21.15.111:0/747487998 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0035020 con 0x7f63f4151740 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63f92fb640 1 -- 172.21.15.111:0/747487998 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f0007540 con 0x7f63f4151740 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63f92fb640 1 -- 172.21.15.111:0/747487998 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4001670 con 0x7f63f4151740 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 -- 172.21.15.111:0/747487998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4151740 msgr2=0x7f63f4071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 --2- 172.21.15.111:0/747487998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4151740 0x7f63f4071600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009dc0 tx=0x7f63f002a640 comp rx=0 tx=0).stop 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 -- 172.21.15.111:0/747487998 shutdown_connections 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 --2- 172.21.15.111:0/747487998 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4151740 0x7f63f4071600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 -- 172.21.15.111:0/747487998 >> 172.21.15.111:0/747487998 conn(0x7f63f40b1c00 msgr2=0x7f63f40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 -- 172.21.15.111:0/747487998 shutdown_connections 2024-05-27T00:40:08.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.662+0000 7f63fc561640 1 -- 172.21.15.111:0/747487998 wait complete. 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.663+0000 7f63fc561640 1 Processor -- start 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 -- start start 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f406c330 0x7f63f4145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4071b40 0x7f63f4155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 0x7f63f41564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f63f415b090 con 0x7f63f4151740 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f63f415af10 con 0x7f63f406c330 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63fc561640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f63f40b42f0 con 0x7f63f4071b40 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63faafe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 0x7f63f41564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63faafe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 0x7f63f41564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42362/0 (socket says 172.21.15.111:42362) 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.664+0000 7f63faafe640 1 -- 172.21.15.111:0/2120072828 learned_addr learned my addr 172.21.15.111:0/2120072828 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:08.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63f9afc640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4071b40 0x7f63f4155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63fa2fd640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f406c330 0x7f63f4145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63faafe640 1 -- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4071b40 msgr2=0x7f63f4155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63faafe640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4071b40 0x7f63f4155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63faafe640 1 -- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f406c330 msgr2=0x7f63f4145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63faafe640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f406c330 0x7f63f4145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63faafe640 1 -- 172.21.15.111:0/2120072828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f41558a0 con 0x7f63f4151740 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63faafe640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 0x7f63f41564f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f63ec001eb0 tx=0x7f63ec00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec015020 con 0x7f63f4151740 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ec0035f0 con 0x7f63f4151740 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0039e0 con 0x7f63f4151740 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f41f40b0 con 0x7f63f4151740 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.665+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f41f4670 con 0x7f63f4151740 2024-05-27T00:40:08.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.666+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f4068680 con 0x7f63f4151740 2024-05-27T00:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.666+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63ec018020 con 0x7f63f4151740 2024-05-27T00:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.667+0000 7f63eaffd640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63d806be30 0x7f63d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.667+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f63ec08cee0 con 0x7f63f4151740 2024-05-27T00:40:08.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.667+0000 7f63fa2fd640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63d806be30 0x7f63d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.668+0000 7f63fa2fd640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63d806be30 0x7f63d806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63f0033000 tx=0x7f63f002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.671+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f63ec01f020 con 0x7f63f4151740 2024-05-27T00:40:08.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.842+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f63f4071620 con 0x7f63f4151740 2024-05-27T00:40:08.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.842+0000 7f63eaffd640 1 -- 172.21.15.111:0/2120072828 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f63ec021300 con 0x7f63f4151740 2024-05-27T00:40:08.843 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:08.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63d806be30 msgr2=0x7f63d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63d806be30 0x7f63d806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63f0033000 tx=0x7f63f002f000 comp rx=0 tx=0).stop 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 msgr2=0x7f63f41564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 0x7f63f41564f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f63ec001eb0 tx=0x7f63ec00c870 comp rx=0 tx=0).stop 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 shutdown_connections 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f63d806be30 0x7f63d806e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63f4151740 0x7f63f41564f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f63f4071b40 0x7f63f4155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 --2- 172.21.15.111:0/2120072828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63f406c330 0x7f63f4145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 >> 172.21.15.111:0/2120072828 conn(0x7f63f40b1c00 msgr2=0x7f63f40d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 shutdown_connections 2024-05-27T00:40:08.846 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.844+0000 7f63fc561640 1 -- 172.21.15.111:0/2120072828 wait complete. 2024-05-27T00:40:08.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:08.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.958+0000 7f89dc334640 1 Processor -- start 2024-05-27T00:40:08.959 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.958+0000 7f89dc334640 1 -- start start 2024-05-27T00:40:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.959+0000 7f89dc334640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89d40b3630 0x7f89d40b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.959+0000 7f89dc334640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b77e0 0x7f89d4161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.959+0000 7f89dc334640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89d406cc70 con 0x7f89d4162390 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.959+0000 7f89dc334640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f89d40d1e50 con 0x7f89d40b3630 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.959+0000 7f89dc334640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f89d40d2130 con 0x7f89d40b77e0 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da0d0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f89d40b3630 0x7f89d40b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da8d1640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f89d4162390 0x7f89d40d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41942/0 (socket says 172.21.15.111:41942) 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da8d1640 1 -- 172.21.15.111:0/1294194262 learned_addr learned my addr 172.21.15.111:0/1294194262 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89d98cf640 1 --2- 172.21.15.111:0/1294194262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b77e0 0x7f89d4161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da0d0640 1 -- 172.21.15.111:0/1294194262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b77e0 msgr2=0x7f89d4161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da0d0640 1 --2- 172.21.15.111:0/1294194262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b77e0 0x7f89d4161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da0d0640 1 -- 172.21.15.111:0/1294194262 >> v1:172.21.15.111:6789/0 conn(0x7f89d4162390 legacy=0x7f89d40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89d90ce640 1 -- 172.21.15.111:0/1294194262 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431579409 0 0) 0x7f89d406cc70 con 0x7f89d4162390 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.960+0000 7f89da0d0640 1 -- 172.21.15.111:0/1294194262 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d40d2410 con 0x7f89d40b3630 2024-05-27T00:40:08.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89da0d0640 1 --2- 172.21.15.111:0/1294194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b3630 0x7f89d40b72a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89d00097a0 tx=0x7f89d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cb61e977e78d8f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89d90ce640 1 -- 172.21.15.111:0/1294194262 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0037070 con 0x7f89d40b3630 2024-05-27T00:40:08.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89d90ce640 1 -- 172.21.15.111:0/1294194262 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d0032440 con 0x7f89d40b3630 2024-05-27T00:40:08.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89d90ce640 1 -- 172.21.15.111:0/1294194262 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0032810 con 0x7f89d40b3630 2024-05-27T00:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 -- 172.21.15.111:0/1294194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b3630 msgr2=0x7f89d40b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 --2- 172.21.15.111:0/1294194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b3630 0x7f89d40b72a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89d00097a0 tx=0x7f89d002a180 comp rx=0 tx=0).stop 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 -- 172.21.15.111:0/1294194262 shutdown_connections 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 --2- 172.21.15.111:0/1294194262 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b77e0 0x7f89d4161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 --2- 172.21.15.111:0/1294194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b3630 0x7f89d40b72a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 -- 172.21.15.111:0/1294194262 >> 172.21.15.111:0/1294194262 conn(0x7f89d40d42c0 msgr2=0x7f89d40d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 -- 172.21.15.111:0/1294194262 shutdown_connections 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.961+0000 7f89dc334640 1 -- 172.21.15.111:0/1294194262 wait complete. 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.962+0000 7f89dc334640 1 Processor -- start 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 -- start start 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b3630 0x7f89d4159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 0x7f89d415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89d4162390 0x7f89d40b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40d34a0 con 0x7f89d40b77e0 2024-05-27T00:40:08.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40d3320 con 0x7f89d4162390 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89dc334640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f89d40d3620 con 0x7f89d40b3630 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89d98cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 0x7f89d415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89d98cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 0x7f89d415cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42378/0 (socket says 172.21.15.111:42378) 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.963+0000 7f89d98cf640 1 -- 172.21.15.111:0/1595577648 learned_addr learned my addr 172.21.15.111:0/1595577648 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89da0d0640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b3630 0x7f89d4159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89da8d1640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89d4162390 0x7f89d40b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89d98cf640 1 -- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b3630 msgr2=0x7f89d4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89d98cf640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b3630 0x7f89d4159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89d98cf640 1 -- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89d4162390 msgr2=0x7f89d40b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89d98cf640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89d4162390 0x7f89d40b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89d98cf640 1 -- 172.21.15.111:0/1595577648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d415b490 con 0x7f89d40b77e0 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89da8d1640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89d4162390 0x7f89d40b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:08.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89d98cf640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 0x7f89d415cc70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c820 tx=0x7f89c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4015070 con 0x7f89d40b77e0 2024-05-27T00:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c4004240 con 0x7f89d40b77e0 2024-05-27T00:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c400ce10 con 0x7f89d40b77e0 2024-05-27T00:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d415b770 con 0x7f89d40b77e0 2024-05-27T00:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.964+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d415bd30 con 0x7f89d40b77e0 2024-05-27T00:40:08.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.965+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d4068680 con 0x7f89d40b77e0 2024-05-27T00:40:08.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.965+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89c4007830 con 0x7f89d40b77e0 2024-05-27T00:40:08.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.966+0000 7f89caffd640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89b806be30 0x7f89b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:08.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.966+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f89c408b0e0 con 0x7f89d40b77e0 2024-05-27T00:40:08.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.966+0000 7f89da0d0640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89b806be30 0x7f89b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:08.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.966+0000 7f89da0d0640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89b806be30 0x7f89b806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f89d0034000 tx=0x7f89d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:08.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:08.970+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f89c4059700 con 0x7f89d40b77e0 2024-05-27T00:40:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.143+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f89d41596c0 con 0x7f89d40b77e0 2024-05-27T00:40:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.143+0000 7f89caffd640 1 -- 172.21.15.111:0/1595577648 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f89c4059000 con 0x7f89d40b77e0 2024-05-27T00:40:09.145 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:09.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89b806be30 msgr2=0x7f89b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89b806be30 0x7f89b806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f89d0034000 tx=0x7f89d002e000 comp rx=0 tx=0).stop 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 msgr2=0x7f89d415cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 0x7f89d415cc70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c820 tx=0x7f89c400cce0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 shutdown_connections 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89b806be30 0x7f89b806e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89d4162390 0x7f89d40b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89d40b77e0 0x7f89d415cc70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 --2- 172.21.15.111:0/1595577648 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f89d40b3630 0x7f89d4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.145+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 >> 172.21.15.111:0/1595577648 conn(0x7f89d40d42c0 msgr2=0x7f89d40d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.146+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 shutdown_connections 2024-05-27T00:40:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.146+0000 7f89dc334640 1 -- 172.21.15.111:0/1595577648 wait complete. 2024-05-27T00:40:09.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T00:40:09.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.257+0000 7f54d5ae1640 1 Processor -- start 2024-05-27T00:40:09.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.258+0000 7f54d5ae1640 1 -- start start 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.258+0000 7f54d5ae1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f54d00d3320 0x7f54d01622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.258+0000 7f54d5ae1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d006cc70 con 0x7f54d0161e50 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.258+0000 7f54d5ae1640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d00d1e50 con 0x7f54d00b3630 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.258+0000 7f54d5ae1640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f54d00d2130 con 0x7f54d00d3320 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.258+0000 7f54ceffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f54d0161e50 0x7f54d00d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41946/0 (socket says 172.21.15.111:41946) 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 -- 172.21.15.111:0/2193212053 learned_addr learned my addr 172.21.15.111:0/2193212053 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539220812 0 0) 0x7f54d006cc70 con 0x7f54d0161e50 2024-05-27T00:40:09.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54cffff640 1 --2- 172.21.15.111:0/2193212053 >> v2:172.21.15.153:3300/0 conn(0x7f54d00d3320 0x7f54d01622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54b8003520 con 0x7f54d0161e50 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 843490475 0 0) 0x7f54b8003520 con 0x7f54d0161e50 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 >> v2:172.21.15.153:3300/0 conn(0x7f54d00d3320 msgr2=0x7f54d01622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 --2- 172.21.15.111:0/2193212053 >> v2:172.21.15.153:3300/0 conn(0x7f54d00d3320 0x7f54d01622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 >> v1:172.21.15.153:6789/0 conn(0x7f54d00b3630 legacy=0x7f54d00b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d00d2410 con 0x7f54d0161e50 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f54c4001670 con 0x7f54d0161e50 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 >> v1:172.21.15.111:6789/0 conn(0x7f54d0161e50 legacy=0x7f54d00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b80039b0 0x7f54b8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8004800 0x7f54b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54b80051c0 0x7f54b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54d00d2130 con 0x7f54b8004800 2024-05-27T00:40:09.263 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54b800c0f0 con 0x7f54b80039b0 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f54b800c2f0 con 0x7f54b80051c0 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8004800 0x7f54b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54cf7fe640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b80039b0 0x7f54b8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54cffff640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54b80051c0 0x7f54b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 -- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54b80051c0 msgr2=0x7f54b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54b80051c0 0x7f54b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 -- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b80039b0 msgr2=0x7f54b8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b80039b0 0x7f54b8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54ceffd640 1 -- 172.21.15.111:0/2193212053 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b8003520 con 0x7f54b8004800 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54cf7fe640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b80039b0 0x7f54b8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.259+0000 7f54cffff640 1 -- 172.21.15.111:0/2193212053 reap_dead start 2024-05-27T00:40:09.264 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.260+0000 7f54ceffd640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8004800 0x7f54b8004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f54c4007fd0 tx=0x7f54b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f99b7f6cb93fb0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.260+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4007d50 con 0x7f54b8004800 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.260+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4005af0 con 0x7f54b8004800 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.260+0000 7f54ce7fc640 1 -- 172.21.15.111:0/2193212053 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4007070 con 0x7f54b8004800 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 -- 172.21.15.111:0/2193212053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8004800 msgr2=0x7f54b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8004800 0x7f54b8004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f54c4007fd0 tx=0x7f54b8005830 comp rx=0 tx=0).stop 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 -- 172.21.15.111:0/2193212053 shutdown_connections 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/2193212053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54b8004800 0x7f54b8004c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 -- 172.21.15.111:0/2193212053 >> 172.21.15.111:0/2193212053 conn(0x7f54d00d42c0 msgr2=0x7f54d00d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 -- 172.21.15.111:0/2193212053 shutdown_connections 2024-05-27T00:40:09.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.261+0000 7f54d5ae1640 1 -- 172.21.15.111:0/2193212053 wait complete. 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.262+0000 7f54d5ae1640 1 Processor -- start 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.262+0000 7f54d5ae1640 1 -- start start 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54d5ae1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b8004800 0x7f54d015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54d5ae1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54d00cdd80 0x7f54d00ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54d5ae1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 0x7f54d00b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54d5ae1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0162460 con 0x7f54d00ccb30 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54d5ae1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01622e0 con 0x7f54b8004800 2024-05-27T00:40:09.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54d5ae1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f54d01625e0 con 0x7f54d00cdd80 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 0x7f54d00b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 0x7f54d00b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42392/0 (socket says 172.21.15.111:42392) 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 -- 172.21.15.111:0/940350124 learned_addr learned my addr 172.21.15.111:0/940350124 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cf7fe640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b8004800 0x7f54d015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54ceffd640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54d00cdd80 0x7f54d00ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 -- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54d00cdd80 msgr2=0x7f54d00ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54d00cdd80 0x7f54d00ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 -- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b8004800 msgr2=0x7f54d015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b8004800 0x7f54d015cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 -- 172.21.15.111:0/940350124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d00ac970 con 0x7f54d00ccb30 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cf7fe640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b8004800 0x7f54d015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.263+0000 7f54cffff640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 0x7f54d00b7260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54d00d3c00 tx=0x7f54c8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:09.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.264+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8014410 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.264+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c8019070 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.264+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d00acc10 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.264+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8014810 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.264+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d00cc330 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.265+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d0068680 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.265+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54c80149b0 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.265+0000 7f54b7fff640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54bc06be30 0x7f54bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.265+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f54c808b880 con 0x7f54d00ccb30 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.266+0000 7f54cf7fe640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54bc06be30 0x7f54bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.266+0000 7f54cf7fe640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54bc06be30 0x7f54bc06e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007fd0 tx=0x7f54c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:09.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.270+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f54c8059ea0 con 0x7f54d00ccb30 2024-05-27T00:40:09.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.452+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f54d00ce470 con 0x7f54d00ccb30 2024-05-27T00:40:09.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.452+0000 7f54b7fff640 1 -- 172.21.15.111:0/940350124 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f54c80597a0 con 0x7f54d00ccb30 2024-05-27T00:40:09.453 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T00:40:09.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54bc06be30 msgr2=0x7f54bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54bc06be30 0x7f54bc06e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007fd0 tx=0x7f54c0009040 comp rx=0 tx=0).stop 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 msgr2=0x7f54d00b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 0x7f54d00b7260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54d00d3c00 tx=0x7f54c8007700 comp rx=0 tx=0).stop 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 shutdown_connections 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f54bc06be30 0x7f54bc06e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d00ccb30 0x7f54d00b7260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f54d00cdd80 0x7f54d00ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 --2- 172.21.15.111:0/940350124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54b8004800 0x7f54d015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 >> 172.21.15.111:0/940350124 conn(0x7f54d00d42c0 msgr2=0x7f54d00b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 shutdown_connections 2024-05-27T00:40:09.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.454+0000 7f54d5ae1640 1 -- 172.21.15.111:0/940350124 wait complete. 2024-05-27T00:40:09.466 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T00:40:09.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:09.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-27T00:40:09.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:40:09.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:09.493 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T00:40:09.549 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:40:09.550 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:09.569 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:40:09.570 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:09.628 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:40:09.628 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:09.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:09.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.784+0000 7fef76de7640 1 Processor -- start 2024-05-27T00:40:09.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.784+0000 7fef76de7640 1 -- start start 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef76de7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fef70151740 0x7fef70071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef76de7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef70071fe0 con 0x7fef7006c330 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef76de7640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef700722c0 con 0x7fef70071b40 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef76de7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fef700725a0 con 0x7fef70151740 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef74b83640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fef70151740 0x7fef70071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef75384640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fef7006c330 0x7fef700cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41954/0 (socket says 172.21.15.111:41954) 2024-05-27T00:40:09.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.785+0000 7fef75384640 1 -- 172.21.15.111:0/2463956006 learned_addr learned my addr 172.21.15.111:0/2463956006 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef74b83640 1 -- 172.21.15.111:0/2463956006 >> v1:172.21.15.153:6789/0 conn(0x7fef70071b40 legacy=0x7fef7006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef74b83640 1 -- 172.21.15.111:0/2463956006 >> v1:172.21.15.111:6789/0 conn(0x7fef7006c330 legacy=0x7fef700cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef74b83640 1 -- 172.21.15.111:0/2463956006 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef70072880 con 0x7fef70151740 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef6f7fe640 1 -- 172.21.15.111:0/2463956006 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106933642 0 0) 0x7fef70071fe0 con 0x7fef7006c330 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef74b83640 1 --2- 172.21.15.111:0/2463956006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef70071600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef600097a0 tx=0x7fef6002b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc344e13cb3ba07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef6f7fe640 1 -- 172.21.15.111:0/2463956006 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60038070 con 0x7fef70151740 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef6f7fe640 1 -- 172.21.15.111:0/2463956006 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60033440 con 0x7fef70151740 2024-05-27T00:40:09.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.786+0000 7fef6f7fe640 1 -- 172.21.15.111:0/2463956006 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60033810 con 0x7fef70151740 2024-05-27T00:40:09.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 -- 172.21.15.111:0/2463956006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 msgr2=0x7fef70071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 --2- 172.21.15.111:0/2463956006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef70071600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef600097a0 tx=0x7fef6002b2f0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 -- 172.21.15.111:0/2463956006 shutdown_connections 2024-05-27T00:40:09.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 --2- 172.21.15.111:0/2463956006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef70071600 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 -- 172.21.15.111:0/2463956006 >> 172.21.15.111:0/2463956006 conn(0x7fef700b1c00 msgr2=0x7fef700b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:09.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 -- 172.21.15.111:0/2463956006 shutdown_connections 2024-05-27T00:40:09.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.787+0000 7fef76de7640 1 -- 172.21.15.111:0/2463956006 wait complete. 2024-05-27T00:40:09.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.788+0000 7fef76de7640 1 Processor -- start 2024-05-27T00:40:09.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.788+0000 7fef76de7640 1 -- start start 2024-05-27T00:40:09.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef76de7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef7006c330 0x7fef70157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef76de7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef70071b40 0x7fef701341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef76de7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef700cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef76de7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef7015b090 con 0x7fef70151740 2024-05-27T00:40:09.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef76de7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fef7015af10 con 0x7fef7006c330 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef76de7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fef700b42f0 con 0x7fef70071b40 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef700cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef700cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42410/0 (socket says 172.21.15.111:42410) 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 -- 172.21.15.111:0/449989563 learned_addr learned my addr 172.21.15.111:0/449989563 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef74b83640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef7006c330 0x7fef70157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef6ffff640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef70071b40 0x7fef701341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 -- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef70071b40 msgr2=0x7fef701341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef70071b40 0x7fef701341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 -- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef7006c330 msgr2=0x7fef70157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef7006c330 0x7fef70157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.789+0000 7fef75384640 1 -- 172.21.15.111:0/449989563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701549e0 con 0x7fef70151740 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.790+0000 7fef75384640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef700cd490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c450 tx=0x7fef6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:09.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.790+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68018070 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.790+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68007500 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.790+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef701554d0 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.790+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68004a20 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.790+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef700cde40 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.791+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef700ce040 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.791+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef6801b070 con 0x7fef70151740 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.792+0000 7fef6d7fa640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef5c06be30 0x7fef5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:09.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.792+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fef6808a440 con 0x7fef70151740 2024-05-27T00:40:09.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.792+0000 7fef74b83640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef5c06be30 0x7fef5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:09.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.792+0000 7fef74b83640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef5c06be30 0x7fef5c06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef60035000 tx=0x7fef6002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:09.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.796+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fef68058410 con 0x7fef70151740 2024-05-27T00:40:09.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.970+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef70068680 con 0x7fef70151740 2024-05-27T00:40:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.971+0000 7fef6d7fa640 1 -- 172.21.15.111:0/449989563 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7fef6805c710 con 0x7fef70151740 2024-05-27T00:40:09.972 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:09.972 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef5c06be30 msgr2=0x7fef5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef5c06be30 0x7fef5c06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef60035000 tx=0x7fef6002f000 comp rx=0 tx=0).stop 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 msgr2=0x7fef700cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef700cd490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c450 tx=0x7fef6800c910 comp rx=0 tx=0).stop 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 shutdown_connections 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fef5c06be30 0x7fef5c06e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef70151740 0x7fef700cd490 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fef70071b40 0x7fef701341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 --2- 172.21.15.111:0/449989563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef7006c330 0x7fef70157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 >> 172.21.15.111:0/449989563 conn(0x7fef700b1c00 msgr2=0x7fef70071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 shutdown_connections 2024-05-27T00:40:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:09.973+0000 7fef76de7640 1 -- 172.21.15.111:0/449989563 wait complete. 2024-05-27T00:40:09.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:09.986 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:10.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.091+0000 7f8872083640 1 Processor -- start 2024-05-27T00:40:10.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.091+0000 7f8872083640 1 -- start start 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8872083640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f886c144000 0x7f886c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8872083640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f886c06b390 0x7f886c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8872083640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f886c0b3440 con 0x7f886c0cdc80 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8872083640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f886c0b42f0 con 0x7f886c06b390 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8872083640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f886c0b36e0 con 0x7f886c144000 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f886c06b390 0x7f886c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f886c06b390 0x7f886c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42412/0 (socket says 172.21.15.111:42412) 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 -- 172.21.15.111:0/2207158025 learned_addr learned my addr 172.21.15.111:0/2207158025 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f886b7fe640 1 --2- 172.21.15.111:0/2207158025 >> v2:172.21.15.153:3300/0 conn(0x7f886c144000 0x7f886c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 -- 172.21.15.111:0/2207158025 >> v2:172.21.15.153:3300/0 conn(0x7f886c144000 msgr2=0x7f886c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:10.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 --2- 172.21.15.111:0/2207158025 >> v2:172.21.15.153:3300/0 conn(0x7f886c144000 0x7f886c0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 -- 172.21.15.111:0/2207158025 >> v1:172.21.15.153:6789/0 conn(0x7f886c0cdc80 legacy=0x7f886c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.092+0000 7f8863fff640 1 -- 172.21.15.111:0/2207158025 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c0b39c0 con 0x7f886c06b390 2024-05-27T00:40:10.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.093+0000 7f8863fff640 1 --2- 172.21.15.111:0/2207158025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c06b390 0x7f886c0cd740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f886c0726a0 tx=0x7f885402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198f96155d4ebe96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.093+0000 7f886affd640 1 -- 172.21.15.111:0/2207158025 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854037070 con 0x7f886c06b390 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.093+0000 7f886affd640 1 -- 172.21.15.111:0/2207158025 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8854032440 con 0x7f886c06b390 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.093+0000 7f886affd640 1 -- 172.21.15.111:0/2207158025 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854032810 con 0x7f886c06b390 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 -- 172.21.15.111:0/2207158025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c06b390 msgr2=0x7f886c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 --2- 172.21.15.111:0/2207158025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c06b390 0x7f886c0cd740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f886c0726a0 tx=0x7f885402a270 comp rx=0 tx=0).stop 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 -- 172.21.15.111:0/2207158025 shutdown_connections 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 --2- 172.21.15.111:0/2207158025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c06b390 0x7f886c0cd740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 --2- 172.21.15.111:0/2207158025 >> v2:172.21.15.153:3300/0 conn(0x7f886c144000 0x7f886c0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 -- 172.21.15.111:0/2207158025 >> 172.21.15.111:0/2207158025 conn(0x7f886c0b1c00 msgr2=0x7f886c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:10.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 -- 172.21.15.111:0/2207158025 shutdown_connections 2024-05-27T00:40:10.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.094+0000 7f8872083640 1 -- 172.21.15.111:0/2207158025 wait complete. 2024-05-27T00:40:10.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.095+0000 7f8872083640 1 Processor -- start 2024-05-27T00:40:10.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.095+0000 7f8872083640 1 -- start start 2024-05-27T00:40:10.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8872083640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f886c06b390 0x7f886c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8872083640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f886c0cdc80 0x7f886c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8872083640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 0x7f886c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8872083640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f886c072de0 con 0x7f886c144000 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8872083640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f886c0b3f80 con 0x7f886c06b390 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8872083640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f886c072f60 con 0x7f886c0cdc80 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 0x7f886c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 0x7f886c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42426/0 (socket says 172.21.15.111:42426) 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 -- 172.21.15.111:0/127614183 learned_addr learned my addr 172.21.15.111:0/127614183 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886b7fe640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f886c06b390 0x7f886c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8863fff640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f886c0cdc80 0x7f886c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 -- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f886c0cdc80 msgr2=0x7f886c14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f886c0cdc80 0x7f886c14e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 -- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f886c06b390 msgr2=0x7f886c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f886c06b390 0x7f886c0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 -- 172.21.15.111:0/127614183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c14ef00 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f8863fff640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f886c0cdc80 0x7f886c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.096+0000 7f886bfff640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 0x7f886c151c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f886400c7d0 tx=0x7f886400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.097+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864003b70 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.097+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8864003d30 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.097+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886400aa90 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.097+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c1f5120 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.097+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c1f56e0 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.098+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c068680 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.098+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f886400ac30 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.099+0000 7f8868ff9640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f884c06be30 0x7f884c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.099+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f886408b060 con 0x7f886c144000 2024-05-27T00:40:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.099+0000 7f886b7fe640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f884c06be30 0x7f884c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.099+0000 7f886b7fe640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f884c06be30 0x7f884c06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f885c009b40 tx=0x7f885c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.103+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f886c14ef00 con 0x7f886c144000 2024-05-27T00:40:10.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.274+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f886c071ca0 con 0x7f886c144000 2024-05-27T00:40:10.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.275+0000 7f8868ff9640 1 -- 172.21.15.111:0/127614183 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f8864059680 con 0x7f886c144000 2024-05-27T00:40:10.276 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f884c06be30 msgr2=0x7f884c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f884c06be30 0x7f884c06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f885c009b40 tx=0x7f885c009190 comp rx=0 tx=0).stop 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 msgr2=0x7f886c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 0x7f886c151c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f886400c7d0 tx=0x7f886400cc90 comp rx=0 tx=0).stop 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 shutdown_connections 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f884c06be30 0x7f884c06e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f886c144000 0x7f886c151c20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f886c0cdc80 0x7f886c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 --2- 172.21.15.111:0/127614183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f886c06b390 0x7f886c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 >> 172.21.15.111:0/127614183 conn(0x7f886c0b1c00 msgr2=0x7f886c13fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:10.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 shutdown_connections 2024-05-27T00:40:10.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.277+0000 7f8872083640 1 -- 172.21.15.111:0/127614183 wait complete. 2024-05-27T00:40:10.289 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.390+0000 7f96875e7640 1 Processor -- start 2024-05-27T00:40:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.390+0000 7f96875e7640 1 -- start start 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f96875e7640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f968006b390 0x7f96800cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f96875e7640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f96800cdc80 0x7f96800726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f96875e7640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96800b36e0 con 0x7f9680144000 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f96875e7640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f96800b42f0 con 0x7f96800cdc80 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f96875e7640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f96800b39c0 con 0x7f968006b390 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685b84640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f96800cdc80 0x7f96800726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685383640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9680144000 0x7f96800b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41960/0 (socket says 172.21.15.111:41960) 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685b84640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f96800cdc80 0x7f96800726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42442/0 (socket says 172.21.15.111:42442) 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685383640 1 -- 172.21.15.111:0/2780148859 learned_addr learned my addr 172.21.15.111:0/2780148859 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9684b82640 1 --2- 172.21.15.111:0/2780148859 >> v2:172.21.15.153:3300/0 conn(0x7f968006b390 0x7f96800cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685b84640 1 -- 172.21.15.111:0/2780148859 >> v2:172.21.15.153:3300/0 conn(0x7f968006b390 msgr2=0x7f96800cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685b84640 1 --2- 172.21.15.111:0/2780148859 >> v2:172.21.15.153:3300/0 conn(0x7f968006b390 0x7f96800cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685b84640 1 -- 172.21.15.111:0/2780148859 >> v1:172.21.15.111:6789/0 conn(0x7f9680144000 legacy=0x7f96800b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.391+0000 7f9685b84640 1 -- 172.21.15.111:0/2780148859 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800b3ca0 con 0x7f96800cdc80 2024-05-27T00:40:10.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.392+0000 7f9685b84640 1 --2- 172.21.15.111:0/2780148859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96800726d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9680074180 tx=0x7f967802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad97b1dc4ddffe7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.394 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.392+0000 7f9677fff640 1 -- 172.21.15.111:0/2780148859 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678039070 con 0x7f96800cdc80 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.392+0000 7f9677fff640 1 -- 172.21.15.111:0/2780148859 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9678009540 con 0x7f96800cdc80 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.392+0000 7f9677fff640 1 -- 172.21.15.111:0/2780148859 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678009910 con 0x7f96800cdc80 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.392+0000 7f96875e7640 1 -- 172.21.15.111:0/2780148859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 msgr2=0x7f96800726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.392+0000 7f96875e7640 1 --2- 172.21.15.111:0/2780148859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96800726d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9680074180 tx=0x7f967802c890 comp rx=0 tx=0).stop 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.393+0000 7f96875e7640 1 -- 172.21.15.111:0/2780148859 shutdown_connections 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.393+0000 7f96875e7640 1 --2- 172.21.15.111:0/2780148859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96800726d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.393+0000 7f96875e7640 1 --2- 172.21.15.111:0/2780148859 >> v2:172.21.15.153:3300/0 conn(0x7f968006b390 0x7f96800cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.393+0000 7f96875e7640 1 -- 172.21.15.111:0/2780148859 >> 172.21.15.111:0/2780148859 conn(0x7f96800b1c00 msgr2=0x7f96800b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.393+0000 7f96875e7640 1 -- 172.21.15.111:0/2780148859 shutdown_connections 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.393+0000 7f96875e7640 1 -- 172.21.15.111:0/2780148859 wait complete. 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 Processor -- start 2024-05-27T00:40:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 -- start start 2024-05-27T00:40:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f968006b390 0x7f968015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96801e6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9680144000 0x7f96801e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9680072d90 con 0x7f96800cdc80 2024-05-27T00:40:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.394+0000 7f96875e7640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9680072c10 con 0x7f9680144000 2024-05-27T00:40:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f96875e7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9680072f10 con 0x7f968006b390 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96801e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96801e6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42450/0 (socket says 172.21.15.111:42450) 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 -- 172.21.15.111:0/3959183235 learned_addr learned my addr 172.21.15.111:0/3959183235 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9685383640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f968006b390 0x7f968015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9685b84640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9680144000 0x7f96801e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 -- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f968006b390 msgr2=0x7f968015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f968006b390 0x7f968015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 -- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9680144000 msgr2=0x7f96801e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9680144000 0x7f96801e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 -- 172.21.15.111:0/3959183235 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96801e8080 con 0x7f96800cdc80 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9684b82640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96801e6710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f967000d800 tx=0x7f967000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670004020 con 0x7f96800cdc80 2024-05-27T00:40:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700041c0 con 0x7f96800cdc80 2024-05-27T00:40:10.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96801e70b0 con 0x7f96800cdc80 2024-05-27T00:40:10.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.395+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670004390 con 0x7f96800cdc80 2024-05-27T00:40:10.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.396+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96801e7670 con 0x7f96800cdc80 2024-05-27T00:40:10.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.396+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9680068680 con 0x7f96800cdc80 2024-05-27T00:40:10.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.397+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f967001b020 con 0x7f96800cdc80 2024-05-27T00:40:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.397+0000 7f9675ffb640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f966406be30 0x7f966406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.397+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f967008af50 con 0x7f96800cdc80 2024-05-27T00:40:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.397+0000 7f9685383640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f966406be30 0x7f966406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.398+0000 7f9685383640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f966406be30 0x7f966406e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f967c007fd0 tx=0x7f967c007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.401+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9670058e90 con 0x7f96800cdc80 2024-05-27T00:40:10.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.574+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96800cf8d0 con 0x7f96800cdc80 2024-05-27T00:40:10.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.575+0000 7f9675ffb640 1 -- 172.21.15.111:0/3959183235 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f967005d190 con 0x7f96800cdc80 2024-05-27T00:40:10.576 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:10.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f966406be30 msgr2=0x7f966406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f966406be30 0x7f966406e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f967c007fd0 tx=0x7f967c007e80 comp rx=0 tx=0).stop 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 msgr2=0x7f96801e6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96801e6710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f967000d800 tx=0x7f967000dcc0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 shutdown_connections 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f966406be30 0x7f966406e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9680144000 0x7f96801e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96800cdc80 0x7f96801e6710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 --2- 172.21.15.111:0/3959183235 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f968006b390 0x7f968015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.577+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 >> 172.21.15.111:0/3959183235 conn(0x7f96800b1c00 msgr2=0x7f96800ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.578+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 shutdown_connections 2024-05-27T00:40:10.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.578+0000 7f96875e7640 1 -- 172.21.15.111:0/3959183235 wait complete. 2024-05-27T00:40:10.590 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.693+0000 7fca78942640 1 Processor -- start 2024-05-27T00:40:10.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.693+0000 7fca78942640 1 -- start start 2024-05-27T00:40:10.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca78942640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fca74144000 0x7fca740b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca78942640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fca7406b390 0x7fca740cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca78942640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca740b3440 con 0x7fca740cdc80 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca78942640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fca740b42f0 con 0x7fca7406b390 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca78942640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fca740b36e0 con 0x7fca74144000 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fca7406b390 0x7fca740cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fca7406b390 0x7fca740cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36240/0 (socket says 172.21.15.111:36240) 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 -- 172.21.15.111:0/2678608233 learned_addr learned my addr 172.21.15.111:0/2678608233 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca7259c640 1 --2- 172.21.15.111:0/2678608233 >> v2:172.21.15.153:3300/0 conn(0x7fca74144000 0x7fca740b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 -- 172.21.15.111:0/2678608233 >> v2:172.21.15.153:3300/0 conn(0x7fca74144000 msgr2=0x7fca740b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 --2- 172.21.15.111:0/2678608233 >> v2:172.21.15.153:3300/0 conn(0x7fca74144000 0x7fca740b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 -- 172.21.15.111:0/2678608233 >> v1:172.21.15.153:6789/0 conn(0x7fca740cdc80 legacy=0x7fca740726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.694+0000 7fca71d9b640 1 -- 172.21.15.111:0/2678608233 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca740b39c0 con 0x7fca7406b390 2024-05-27T00:40:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.695+0000 7fca71d9b640 1 --2- 172.21.15.111:0/2678608233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca7406b390 0x7fca740cd740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fca740726a0 tx=0x7fca5c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772d6c9f31b8c719 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.695+0000 7fca7159a640 1 -- 172.21.15.111:0/2678608233 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0325a0 con 0x7fca7406b390 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.695+0000 7fca7159a640 1 -- 172.21.15.111:0/2678608233 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c038070 con 0x7fca7406b390 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.695+0000 7fca7159a640 1 -- 172.21.15.111:0/2678608233 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c032960 con 0x7fca7406b390 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 -- 172.21.15.111:0/2678608233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca7406b390 msgr2=0x7fca740cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 --2- 172.21.15.111:0/2678608233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca7406b390 0x7fca740cd740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fca740726a0 tx=0x7fca5c02c8e0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 -- 172.21.15.111:0/2678608233 shutdown_connections 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 --2- 172.21.15.111:0/2678608233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca7406b390 0x7fca740cd740 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 --2- 172.21.15.111:0/2678608233 >> v2:172.21.15.153:3300/0 conn(0x7fca74144000 0x7fca740b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 -- 172.21.15.111:0/2678608233 >> 172.21.15.111:0/2678608233 conn(0x7fca740b1c00 msgr2=0x7fca740b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 -- 172.21.15.111:0/2678608233 shutdown_connections 2024-05-27T00:40:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.696+0000 7fca78942640 1 -- 172.21.15.111:0/2678608233 wait complete. 2024-05-27T00:40:10.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.697+0000 7fca78942640 1 Processor -- start 2024-05-27T00:40:10.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.697+0000 7fca78942640 1 -- start start 2024-05-27T00:40:10.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca78942640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7406b390 0x7fca7413c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca78942640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 0x7fca7413f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca78942640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca74144000 0x7fca740aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca78942640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fca74072de0 con 0x7fca740cdc80 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca78942640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fca740b3f80 con 0x7fca74144000 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca78942640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fca74072f60 con 0x7fca7406b390 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 0x7fca7413f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 0x7fca7413f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36242/0 (socket says 172.21.15.111:36242) 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 -- 172.21.15.111:0/3524012891 learned_addr learned my addr 172.21.15.111:0/3524012891 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 -- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7406b390 msgr2=0x7fca7413c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7406b390 0x7fca7413c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 -- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca74144000 msgr2=0x7fca740aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca74144000 0x7fca740aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.698+0000 7fca71d9b640 1 -- 172.21.15.111:0/3524012891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7413f970 con 0x7fca740cdc80 2024-05-27T00:40:10.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.699+0000 7fca71d9b640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 0x7fca7413f430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fca5c02c920 tx=0x7fca5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.699+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c033840 con 0x7fca740cdc80 2024-05-27T00:40:10.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.699+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c033a00 con 0x7fca740cdc80 2024-05-27T00:40:10.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.699+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7413e390 con 0x7fca740cdc80 2024-05-27T00:40:10.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.699+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c033df0 con 0x7fca740cdc80 2024-05-27T00:40:10.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.699+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7413e950 con 0x7fca740cdc80 2024-05-27T00:40:10.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.700+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca74068680 con 0x7fca740cdc80 2024-05-27T00:40:10.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.700+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca5c03c3f0 con 0x7fca740cdc80 2024-05-27T00:40:10.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.701+0000 7fca6affd640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.701+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fca5c030030 con 0x7fca740cdc80 2024-05-27T00:40:10.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.701+0000 7fca7259c640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.701+0000 7fca7259c640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fca5406be30 0x7fca5406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca64007e60 tx=0x7fca64007390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:10.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.705+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fca5c041070 con 0x7fca740cdc80 2024-05-27T00:40:10.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.876+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca74140aa0 con 0x7fca740cdc80 2024-05-27T00:40:10.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.876+0000 7fca6affd640 1 -- 172.21.15.111:0/3524012891 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fca5c07cb00 con 0x7fca740cdc80 2024-05-27T00:40:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fca5406be30 msgr2=0x7fca5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fca5406be30 0x7fca5406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca64007e60 tx=0x7fca64007390 comp rx=0 tx=0).stop 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 msgr2=0x7fca7413f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 0x7fca7413f430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fca5c02c920 tx=0x7fca5c0046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 shutdown_connections 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca74144000 0x7fca740aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fca740cdc80 0x7fca7413f430 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 --2- 172.21.15.111:0/3524012891 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fca7406b390 0x7fca7413c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 >> 172.21.15.111:0/3524012891 conn(0x7fca740b1c00 msgr2=0x7fca74143000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 shutdown_connections 2024-05-27T00:40:10.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.878+0000 7fca78942640 1 -- 172.21.15.111:0/3524012891 wait complete. 2024-05-27T00:40:10.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:10.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.993+0000 7f8373464640 1 Processor -- start 2024-05-27T00:40:10.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.993+0000 7f8373464640 1 -- start start 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8373464640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 0x7f836c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8373464640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c0b3440 con 0x7f836c06b390 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8373464640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c0b36e0 con 0x7f836c0726d0 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8373464640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f836c0b42f0 con 0x7f836c144000 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f83709ff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f836c06b390 0x7f836c0cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:35998/0 (socket says 172.21.15.111:35998) 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f83709ff640 1 -- 172.21.15.111:0/1023065464 learned_addr learned my addr 172.21.15.111:0/1023065464 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8371200640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 0x7f836c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:10.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627706870 0 0) 0x7f836c0b3440 con 0x7f836c06b390 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8358003520 con 0x7f836c06b390 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3088915133 0 0) 0x7f8358003520 con 0x7f836c06b390 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 msgr2=0x7f836c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 0x7f836c0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 >> v1:172.21.15.153:6789/0 conn(0x7f836c0726d0 legacy=0x7f836c0b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c0b39c0 con 0x7f836c06b390 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f835c001670 con 0x7f836c06b390 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.994+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 >> v1:172.21.15.111:6789/0 conn(0x7f836c06b390 legacy=0x7f836c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8363fff640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f8358003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8363fff640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83580043b0 0x7f8358004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8363fff640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 0x7f83580050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f836c0b42f0 con 0x7f83580039e0 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f835800b9f0 con 0x7f83580043b0 2024-05-27T00:40:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f835800bbf0 con 0x7f836c144000 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371200640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f8358003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f83709ff640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83580043b0 0x7f8358004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371a01640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 0x7f83580050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371200640 1 -- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 msgr2=0x7f83580050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371200640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c144000 0x7f83580050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371200640 1 -- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83580043b0 msgr2=0x7f8358004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371200640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83580043b0 0x7f8358004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f8371200640 1 -- 172.21.15.111:0/1023065464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8358003520 con 0x7f83580039e0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.995+0000 7f83709ff640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83580043b0 0x7f8358004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8371200640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f8358003de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8368009ac0 tx=0x7f8358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253b9d9c450b911 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368036070 con 0x7f83580039e0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368031400 con 0x7f83580039e0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8363fff640 1 -- 172.21.15.111:0/1023065464 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368009690 con 0x7f83580039e0 2024-05-27T00:40:11.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8373464640 1 -- 172.21.15.111:0/1023065464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 msgr2=0x7f8358003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8373464640 1 --2- 172.21.15.111:0/1023065464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f8358003de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8368009ac0 tx=0x7f8358005830 comp rx=0 tx=0).stop 2024-05-27T00:40:11.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8371a01640 1 -- 172.21.15.111:0/1023065464 reap_dead start 2024-05-27T00:40:11.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8373464640 1 -- 172.21.15.111:0/1023065464 shutdown_connections 2024-05-27T00:40:11.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8373464640 1 -- 172.21.15.111:0/1023065464 >> 172.21.15.111:0/1023065464 conn(0x7f836c0b1c00 msgr2=0x7f836c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:11.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.996+0000 7f8373464640 1 -- 172.21.15.111:0/1023065464 shutdown_connections 2024-05-27T00:40:11.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.997+0000 7f8373464640 1 -- 172.21.15.111:0/1023065464 wait complete. 2024-05-27T00:40:11.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 Processor -- start 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 -- start start 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f836c14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f836c0aee20 0x7f836c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c155190 0x7f836c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f836c0cde50 con 0x7f83580039e0 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f836c0b3f80 con 0x7f836c0aee20 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8373464640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f836c0b2f00 con 0x7f836c155190 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8371200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f836c14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8371200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f836c14f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36266/0 (socket says 172.21.15.111:36266) 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.998+0000 7f8371200640 1 -- 172.21.15.111:0/1710955733 learned_addr learned my addr 172.21.15.111:0/1710955733 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371a01640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c155190 0x7f836c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f83709ff640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f836c0aee20 0x7f836c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371200640 1 -- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c155190 msgr2=0x7f836c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371200640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c155190 0x7f836c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371200640 1 -- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f836c0aee20 msgr2=0x7f836c0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371200640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f836c0aee20 0x7f836c0b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371200640 1 -- 172.21.15.111:0/1710955733 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c152570 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f83709ff640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f836c0aee20 0x7f836c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8371200640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f836c14f5a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8368009ac0 tx=0x7f8368004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368040020 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368003bc0 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368036040 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c14fae0 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:10.999+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c151cd0 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.000+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836c068680 con 0x7f83580039e0 2024-05-27T00:40:11.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.001+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8368043020 con 0x7f83580039e0 2024-05-27T00:40:11.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.001+0000 7f8361ffb640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f835006be30 0x7f835006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.001+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f83680ab320 con 0x7f83580039e0 2024-05-27T00:40:11.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.001+0000 7f83709ff640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f835006be30 0x7f835006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.002+0000 7f83709ff640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f835006be30 0x7f835006e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f835c001c60 tx=0x7f835c000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.005+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f836804a070 con 0x7f83580039e0 2024-05-27T00:40:11.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.180+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f836c0b0900 con 0x7f83580039e0 2024-05-27T00:40:11.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.181+0000 7f8361ffb640 1 -- 172.21.15.111:0/1710955733 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f83680044e0 con 0x7f83580039e0 2024-05-27T00:40:11.182 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f835006be30 msgr2=0x7f835006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f835006be30 0x7f835006e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f835c001c60 tx=0x7f835c000ed0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 msgr2=0x7f836c14f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f836c14f5a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8368009ac0 tx=0x7f8368004730 comp rx=0 tx=0).stop 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 shutdown_connections 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f835006be30 0x7f835006e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f836c155190 0x7f836c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f836c0aee20 0x7f836c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 --2- 172.21.15.111:0/1710955733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83580039e0 0x7f836c14f5a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 >> 172.21.15.111:0/1710955733 conn(0x7f836c0b1c00 msgr2=0x7f836c0d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:11.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 shutdown_connections 2024-05-27T00:40:11.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.183+0000 7f8373464640 1 -- 172.21.15.111:0/1710955733 wait complete. 2024-05-27T00:40:11.195 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T00:40:11.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.294+0000 7f135ff5a640 1 Processor -- start 2024-05-27T00:40:11.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.295+0000 7f135ff5a640 1 -- start start 2024-05-27T00:40:11.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.295+0000 7f135ff5a640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1358144000 0x7f13580b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.295+0000 7f135ff5a640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f135806b390 0x7f13580cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.295+0000 7f135ff5a640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13580b3440 con 0x7f13580cdc80 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.295+0000 7f135ff5a640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f13580b42f0 con 0x7f135806b390 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.295+0000 7f135ff5a640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f13580b36e0 con 0x7f1358144000 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f135806b390 0x7f13580cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f135806b390 0x7f13580cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36278/0 (socket says 172.21.15.111:36278) 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 -- 172.21.15.111:0/1058357983 learned_addr learned my addr 172.21.15.111:0/1058357983 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135dcf6640 1 --2- 172.21.15.111:0/1058357983 >> v2:172.21.15.153:3300/0 conn(0x7f1358144000 0x7f13580b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 -- 172.21.15.111:0/1058357983 >> v2:172.21.15.153:3300/0 conn(0x7f1358144000 msgr2=0x7f13580b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 --2- 172.21.15.111:0/1058357983 >> v2:172.21.15.153:3300/0 conn(0x7f1358144000 0x7f13580b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 -- 172.21.15.111:0/1058357983 >> v1:172.21.15.153:6789/0 conn(0x7f13580cdc80 legacy=0x7f13580726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135d4f5640 1 -- 172.21.15.111:0/1058357983 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13580b39c0 con 0x7f135806b390 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.296+0000 7f135dcf6640 1 --2- 172.21.15.111:0/1058357983 >> v2:172.21.15.153:3300/0 conn(0x7f1358144000 0x7f13580b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135d4f5640 1 --2- 172.21.15.111:0/1058357983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f135806b390 0x7f13580cd740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13580726a0 tx=0x7f134802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0631e5d1569f06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ccf4640 1 -- 172.21.15.111:0/1058357983 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348037070 con 0x7f135806b390 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ccf4640 1 -- 172.21.15.111:0/1058357983 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1348032440 con 0x7f135806b390 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ccf4640 1 -- 172.21.15.111:0/1058357983 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348032810 con 0x7f135806b390 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 -- 172.21.15.111:0/1058357983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f135806b390 msgr2=0x7f13580cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 --2- 172.21.15.111:0/1058357983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f135806b390 0x7f13580cd740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13580726a0 tx=0x7f134802a230 comp rx=0 tx=0).stop 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 -- 172.21.15.111:0/1058357983 shutdown_connections 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 --2- 172.21.15.111:0/1058357983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f135806b390 0x7f13580cd740 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 --2- 172.21.15.111:0/1058357983 >> v2:172.21.15.153:3300/0 conn(0x7f1358144000 0x7f13580b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 -- 172.21.15.111:0/1058357983 >> 172.21.15.111:0/1058357983 conn(0x7f13580b1c00 msgr2=0x7f13580b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:11.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 -- 172.21.15.111:0/1058357983 shutdown_connections 2024-05-27T00:40:11.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.297+0000 7f135ff5a640 1 -- 172.21.15.111:0/1058357983 wait complete. 2024-05-27T00:40:11.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 Processor -- start 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 -- start start 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135806b390 0x7f13580ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 0x7f13580abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1358144000 0x7f13580c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1358072de0 con 0x7f13580cdc80 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f13580b3f80 con 0x7f135806b390 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135ff5a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1358072f60 con 0x7f1358144000 2024-05-27T00:40:11.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135d4f5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 0x7f13580abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135d4f5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 0x7f13580abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36286/0 (socket says 172.21.15.111:36286) 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.299+0000 7f135d4f5640 1 -- 172.21.15.111:0/4122162103 learned_addr learned my addr 172.21.15.111:0/4122162103 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135dcf6640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135806b390 0x7f13580ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135e4f7640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1358144000 0x7f13580c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135d4f5640 1 -- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1358144000 msgr2=0x7f13580c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135d4f5640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1358144000 0x7f13580c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135d4f5640 1 -- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135806b390 msgr2=0x7f13580ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135d4f5640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135806b390 0x7f13580ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135d4f5640 1 -- 172.21.15.111:0/4122162103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13580cacb0 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135dcf6640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135806b390 0x7f13580ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135d4f5640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 0x7f13580abb40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1348000c00 tx=0x7f13480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348041020 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1348030970 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348037040 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13580caf90 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.300+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13580cb550 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.301+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1358068680 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.302+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1348044020 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.302+0000 7f134e7fc640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f133c06be30 0x7f133c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.302+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f13480abcd0 con 0x7f13580cdc80 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.302+0000 7f135dcf6640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f133c06be30 0x7f133c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.303+0000 7f135dcf6640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f133c06be30 0x7f133c06e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f135400aa30 tx=0x7f1354008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.306+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f134804b030 con 0x7f13580cdc80 2024-05-27T00:40:11.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.492+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1358071f80 con 0x7f13580cdc80 2024-05-27T00:40:11.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.492+0000 7f134e7fc640 1 -- 172.21.15.111:0/4122162103 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f134807a260 con 0x7f13580cdc80 2024-05-27T00:40:11.493 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f133c06be30 msgr2=0x7f133c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f133c06be30 0x7f133c06e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f135400aa30 tx=0x7f1354008040 comp rx=0 tx=0).stop 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 msgr2=0x7f13580abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 0x7f13580abb40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1348000c00 tx=0x7f13480046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 shutdown_connections 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f133c06be30 0x7f133c06e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1358144000 0x7f13580c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13580cdc80 0x7f13580abb40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 --2- 172.21.15.111:0/4122162103 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f135806b390 0x7f13580ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 >> 172.21.15.111:0/4122162103 conn(0x7f13580b1c00 msgr2=0x7f13580b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:11.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 shutdown_connections 2024-05-27T00:40:11.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.494+0000 7f135ff5a640 1 -- 172.21.15.111:0/4122162103 wait complete. 2024-05-27T00:40:11.506 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-27T00:40:11.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:11.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-27T00:40:11.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:40:11.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:11.540 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-27T00:40:11.608 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:11.608 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:11.624 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:11.624 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:11.680 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:11.681 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:11.736 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:11.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.876+0000 7f4878e37640 1 Processor -- start 2024-05-27T00:40:11.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.876+0000 7f4878e37640 1 -- start start 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4878e37640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f48740b3630 0x7f48740b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4878e37640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4874162560 0x7f48740d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4878e37640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f487406cc70 con 0x7f48740b77e0 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4878e37640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f48740d1e50 con 0x7f4874162560 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4878e37640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f48740d2130 con 0x7f48740b3630 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4872d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4874162560 0x7f48740d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f486bfff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f48740b77e0 0x7f4874161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36004/0 (socket says 172.21.15.111:36004) 2024-05-27T00:40:11.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f486bfff640 1 -- 172.21.15.111:0/2873625354 learned_addr learned my addr 172.21.15.111:0/2873625354 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:11.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4872d9d640 1 -- 172.21.15.111:0/2873625354 >> v2:172.21.15.153:3300/0 conn(0x7f48740b3630 msgr2=0x7f48740b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:11.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4872d9d640 1 --2- 172.21.15.111:0/2873625354 >> v2:172.21.15.153:3300/0 conn(0x7f48740b3630 0x7f48740b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4872d9d640 1 -- 172.21.15.111:0/2873625354 >> v1:172.21.15.111:6789/0 conn(0x7f48740b77e0 legacy=0x7f4874161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4872d9d640 1 -- 172.21.15.111:0/2873625354 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48740d2410 con 0x7f4874162560 2024-05-27T00:40:11.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.877+0000 7f4871d9b640 1 -- 172.21.15.111:0/2873625354 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465751491 0 0) 0x7f487406cc70 con 0x7f48740b77e0 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4872d9d640 1 --2- 172.21.15.111:0/2873625354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740d2c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f487408ccb0 tx=0x7f486c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23dcd0245c8a04f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4871d9b640 1 -- 172.21.15.111:0/2873625354 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c007740 con 0x7f4874162560 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4871d9b640 1 -- 172.21.15.111:0/2873625354 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c0078e0 con 0x7f4874162560 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4871d9b640 1 -- 172.21.15.111:0/2873625354 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c007cb0 con 0x7f4874162560 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 -- 172.21.15.111:0/2873625354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 msgr2=0x7f48740d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 --2- 172.21.15.111:0/2873625354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740d2c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f487408ccb0 tx=0x7f486c02a680 comp rx=0 tx=0).stop 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 -- 172.21.15.111:0/2873625354 shutdown_connections 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 --2- 172.21.15.111:0/2873625354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740d2c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 --2- 172.21.15.111:0/2873625354 >> v2:172.21.15.153:3300/0 conn(0x7f48740b3630 0x7f48740b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 -- 172.21.15.111:0/2873625354 >> 172.21.15.111:0/2873625354 conn(0x7f48740d42c0 msgr2=0x7f48740d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 -- 172.21.15.111:0/2873625354 shutdown_connections 2024-05-27T00:40:11.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.878+0000 7f4878e37640 1 -- 172.21.15.111:0/2873625354 wait complete. 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 Processor -- start 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 -- start start 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48740b3630 0x7f487415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48740b77e0 0x7f48740c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f48740d32d0 con 0x7f4874162560 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f48740d3150 con 0x7f48740b3630 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.880+0000 7f4878e37640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f48740d3450 con 0x7f48740b77e0 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36302/0 (socket says 172.21.15.111:36302) 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 -- 172.21.15.111:0/4038288125 learned_addr learned my addr 172.21.15.111:0/4038288125 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f486bfff640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48740b77e0 0x7f48740c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f487259c640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48740b3630 0x7f487415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 -- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48740b77e0 msgr2=0x7f48740c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48740b77e0 0x7f48740c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 -- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48740b3630 msgr2=0x7f487415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48740b3630 0x7f487415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 -- 172.21.15.111:0/4038288125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487415b900 con 0x7f4874162560 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f486bfff640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48740b77e0 0x7f48740c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f487259c640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48740b3630 0x7f487415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4872d9d640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740cdc70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f486c02ac70 tx=0x7f486c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c040070 con 0x7f4874162560 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c031c90 con 0x7f4874162560 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c045680 con 0x7f4874162560 2024-05-27T00:40:11.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487415bbe0 con 0x7f4874162560 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.881+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487415c1a0 con 0x7f4874162560 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.882+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4874068680 con 0x7f4874162560 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.883+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f486c039040 con 0x7f4874162560 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.883+0000 7f486affd640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f485406be30 0x7f485406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.883+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f486c0accc0 con 0x7f4874162560 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.883+0000 7f487259c640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f485406be30 0x7f485406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:11.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.884+0000 7f487259c640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f485406be30 0x7f485406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f486400a870 tx=0x7f4864008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:11.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:11.887+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f486c07b430 con 0x7f4874162560 2024-05-27T00:40:12.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.062+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48740cef10 con 0x7f4874162560 2024-05-27T00:40:12.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.063+0000 7f486affd640 1 -- 172.21.15.111:0/4038288125 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f486c07ad30 con 0x7f4874162560 2024-05-27T00:40:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:12.066 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:12.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f485406be30 msgr2=0x7f485406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f485406be30 0x7f485406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f486400a870 tx=0x7f4864008040 comp rx=0 tx=0).stop 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 msgr2=0x7f48740cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740cdc70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f486c02ac70 tx=0x7f486c004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 shutdown_connections 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f485406be30 0x7f485406e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4874162560 0x7f48740cdc70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f48740b77e0 0x7f48740c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 --2- 172.21.15.111:0/4038288125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48740b3630 0x7f487415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 >> 172.21.15.111:0/4038288125 conn(0x7f48740d42c0 msgr2=0x7f48740d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 shutdown_connections 2024-05-27T00:40:12.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.065+0000 7f4878e37640 1 -- 172.21.15.111:0/4038288125 wait complete. 2024-05-27T00:40:12.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:12.078 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:12.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.179+0000 7fcbaf7d5640 1 Processor -- start 2024-05-27T00:40:12.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.180+0000 7fcbaf7d5640 1 -- start start 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.180+0000 7fcbaf7d5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcba8161e50 0x7fcba81622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.180+0000 7fcbaf7d5640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcba80d2c10 0x7fcba80d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.180+0000 7fcbaf7d5640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcba806cc70 con 0x7fcba80b3630 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.180+0000 7fcbaf7d5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fcba80d35c0 con 0x7fcba8161e50 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.180+0000 7fcbaf7d5640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcba80d1e50 con 0x7fcba80d2c10 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcba8161e50 0x7fcba81622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcba8161e50 0x7fcba81622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36306/0 (socket says 172.21.15.111:36306) 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 -- 172.21.15.111:0/1653807995 learned_addr learned my addr 172.21.15.111:0/1653807995 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 -- 172.21.15.111:0/1653807995 >> v2:172.21.15.153:3300/0 conn(0x7fcba80d2c10 msgr2=0x7fcba80d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbadd72640 1 --2- 172.21.15.111:0/1653807995 >> v2:172.21.15.153:3300/0 conn(0x7fcba80d2c10 0x7fcba80d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 --2- 172.21.15.111:0/1653807995 >> v2:172.21.15.153:3300/0 conn(0x7fcba80d2c10 0x7fcba80d3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 -- 172.21.15.111:0/1653807995 >> v1:172.21.15.153:6789/0 conn(0x7fcba80b3630 legacy=0x7fcba80b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.181+0000 7fcbacd70640 1 -- 172.21.15.111:0/1653807995 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba80d2130 con 0x7fcba8161e50 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbacd70640 1 --2- 172.21.15.111:0/1653807995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba81622e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcba808d480 tx=0x7fcb9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1e054127602415 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcb9ffff640 1 -- 172.21.15.111:0/1653807995 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98037070 con 0x7fcba8161e50 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcb9ffff640 1 -- 172.21.15.111:0/1653807995 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb98032440 con 0x7fcba8161e50 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcb9ffff640 1 -- 172.21.15.111:0/1653807995 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98032810 con 0x7fcba8161e50 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1653807995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 msgr2=0x7fcba81622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1653807995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba81622e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcba808d480 tx=0x7fcb9802a230 comp rx=0 tx=0).stop 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1653807995 shutdown_connections 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1653807995 >> v2:172.21.15.153:3300/0 conn(0x7fcba80d2c10 0x7fcba80d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1653807995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba81622e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1653807995 >> 172.21.15.111:0/1653807995 conn(0x7fcba80d42c0 msgr2=0x7fcba80d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1653807995 shutdown_connections 2024-05-27T00:40:12.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.182+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1653807995 wait complete. 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 Processor -- start 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 -- start start 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcba80b3630 0x7fcba81588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcba80d2c10 0x7fcba8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcba80b79b0 con 0x7fcba8161e50 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcba80d26f0 con 0x7fcba80b3630 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbaf7d5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcba806bfb0 con 0x7fcba80d2c10 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.184+0000 7fcbadd72640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36310/0 (socket says 172.21.15.111:36310) 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 -- 172.21.15.111:0/1238275736 learned_addr learned my addr 172.21.15.111:0/1238275736 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:12.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbad571640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcba80b3630 0x7fcba81588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbacd70640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcba80d2c10 0x7fcba8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 -- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcba80d2c10 msgr2=0x7fcba8156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcba80d2c10 0x7fcba8156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 -- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcba80b3630 msgr2=0x7fcba81588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcba80b3630 0x7fcba81588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 -- 172.21.15.111:0/1238275736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba80c86a0 con 0x7fcba8161e50 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbadd72640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba815c4c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcba000cd70 tx=0x7fcba000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0003b90 con 0x7fcba8161e50 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba0003d30 con 0x7fcba8161e50 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0004240 con 0x7fcba8161e50 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba80c8940 con 0x7fcba8161e50 2024-05-27T00:40:12.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.185+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba80c8f00 con 0x7fcba8161e50 2024-05-27T00:40:12.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.186+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba8068680 con 0x7fcba8161e50 2024-05-27T00:40:12.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.187+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcba00043e0 con 0x7fcba8161e50 2024-05-27T00:40:12.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.187+0000 7fcb9dffb640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcb8c06be60 0x7fcb8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.187+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fcba008bcc0 con 0x7fcba8161e50 2024-05-27T00:40:12.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.187+0000 7fcbad571640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcb8c06be60 0x7fcb8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.188+0000 7fcbad571640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcb8c06be60 0x7fcb8c06e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcba4004ea0 tx=0x7fcba4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.191+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcba005a2e0 con 0x7fcba8161e50 2024-05-27T00:40:12.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.363+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcba80cef10 con 0x7fcba8161e50 2024-05-27T00:40:12.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.363+0000 7fcb9dffb640 1 -- 172.21.15.111:0/1238275736 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fcba0059be0 con 0x7fcba8161e50 2024-05-27T00:40:12.364 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:12.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcb8c06be60 msgr2=0x7fcb8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.366 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcb8c06be60 0x7fcb8c06e320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcba4004ea0 tx=0x7fcba4009040 comp rx=0 tx=0).stop 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 msgr2=0x7fcba815c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba815c4c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcba000cd70 tx=0x7fcba000a660 comp rx=0 tx=0).stop 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 shutdown_connections 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcb8c06be60 0x7fcb8c06e320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcba8161e50 0x7fcba815c4c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcba80d2c10 0x7fcba8156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 --2- 172.21.15.111:0/1238275736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcba80b3630 0x7fcba81588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 >> 172.21.15.111:0/1238275736 conn(0x7fcba80d42c0 msgr2=0x7fcba81602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 shutdown_connections 2024-05-27T00:40:12.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.365+0000 7fcbaf7d5640 1 -- 172.21.15.111:0/1238275736 wait complete. 2024-05-27T00:40:12.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:12.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.479+0000 7f8194fdc640 1 Processor -- start 2024-05-27T00:40:12.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.479+0000 7f8194fdc640 1 -- start start 2024-05-27T00:40:12.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f8194fdc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 0x7f81900cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f8194fdc640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81900b36e0 con 0x7f81900cdc80 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f8194fdc640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81900b39c0 con 0x7f8190144000 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f8194fdc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f81900b42f0 con 0x7f819006b390 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f818ed9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f81900cdc80 0x7f81900726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36010/0 (socket says 172.21.15.111:36010) 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f818ed9d640 1 -- 172.21.15.111:0/2299369533 learned_addr learned my addr 172.21.15.111:0/2299369533 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f818dd9b640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 0x7f81900cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 38451084 0 0) 0x7f81900b36e0 con 0x7f81900cdc80 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.480+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8178003520 con 0x7f81900cdc80 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2494107220 0 0) 0x7f8178003520 con 0x7f81900cdc80 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 msgr2=0x7f81900cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 0x7f81900cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 >> v1:172.21.15.153:6789/0 conn(0x7f8190144000 legacy=0x7f81900b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81900b3ca0 con 0x7f81900cdc80 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f8188001670 con 0x7f81900cdc80 2024-05-27T00:40:12.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 >> v1:172.21.15.111:6789/0 conn(0x7f81900cdc80 legacy=0x7f81900726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 0x7f81780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8178003f60 0x7f8178004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f81780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81900b42f0 con 0x7f8178004d40 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f817800baa0 con 0x7f8178003f60 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f817800bca0 con 0x7f819006b390 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818ed9d640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f81780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818e59c640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 0x7f81780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818dd9b640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8178003f60 0x7f8178004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818ed9d640 1 -- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 msgr2=0x7f81780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818ed9d640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f819006b390 0x7f81780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818ed9d640 1 -- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8178003f60 msgr2=0x7f8178004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818ed9d640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8178003f60 0x7f8178004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.481+0000 7f818ed9d640 1 -- 172.21.15.111:0/2299369533 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8178003520 con 0x7f8178004d40 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f818ed9d640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f81780051c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f819006ce00 tx=0x7f8178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c686b72152aa2eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188007d10 con 0x7f8178004d40 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8188005ad0 con 0x7f8178004d40 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f818d59a640 1 -- 172.21.15.111:0/2299369533 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188007070 con 0x7f8178004d40 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f8194fdc640 1 -- 172.21.15.111:0/2299369533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 msgr2=0x7f81780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f8194fdc640 1 --2- 172.21.15.111:0/2299369533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f81780051c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f819006ce00 tx=0x7f8178005830 comp rx=0 tx=0).stop 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.482+0000 7f818ed9d640 1 -- 172.21.15.111:0/2299369533 reap_dead start 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.483+0000 7f8194fdc640 1 -- 172.21.15.111:0/2299369533 shutdown_connections 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.483+0000 7f8194fdc640 1 -- 172.21.15.111:0/2299369533 >> 172.21.15.111:0/2299369533 conn(0x7f81900b1c00 msgr2=0x7f81900b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.483+0000 7f8194fdc640 1 -- 172.21.15.111:0/2299369533 shutdown_connections 2024-05-27T00:40:12.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.483+0000 7f8194fdc640 1 -- 172.21.15.111:0/2299369533 wait complete. 2024-05-27T00:40:12.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.484+0000 7f8194fdc640 1 Processor -- start 2024-05-27T00:40:12.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.484+0000 7f8194fdc640 1 -- start start 2024-05-27T00:40:12.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f8194fdc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f819013f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.489 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f8194fdc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8190153c80 0x7f819013e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f8194fdc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819013e9e0 0x7f8190157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f8194fdc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8190072f60 con 0x7f8178004d40 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f8194fdc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8190072de0 con 0x7f819013e9e0 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f8194fdc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f81900cd740 con 0x7f8190153c80 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f819013f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f819013f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36326/0 (socket says 172.21.15.111:36326) 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 -- 172.21.15.111:0/3140809378 learned_addr learned my addr 172.21.15.111:0/3140809378 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 -- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8190153c80 msgr2=0x7f819013e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818ed9d640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819013e9e0 0x7f8190157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818dd9b640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8190153c80 0x7f819013e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8190153c80 0x7f819013e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 -- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819013e9e0 msgr2=0x7f8190157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819013e9e0 0x7f8190157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 -- 172.21.15.111:0/3140809378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819013ee60 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818ed9d640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819013e9e0 0x7f8190157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f818e59c640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f819013f9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f818000fb90 tx=0x7f818000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.485+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000fe90 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.486+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8180018070 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.486+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180003970 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.486+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81900afb30 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.486+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81900b00f0 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.486+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8190068680 con 0x7f8178004d40 2024-05-27T00:40:12.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.487+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8180003b10 con 0x7f8178004d40 2024-05-27T00:40:12.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.488+0000 7f817effd640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f817006bfd0 0x7f817006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.488+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f818008b560 con 0x7f8178004d40 2024-05-27T00:40:12.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.488+0000 7f818dd9b640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f817006bfd0 0x7f817006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.488+0000 7f818dd9b640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f817006bfd0 0x7f817006e490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8190141bf0 tx=0x7f8184008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.493 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.492+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f819013ee60 con 0x7f8178004d40 2024-05-27T00:40:12.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.664+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8190071fc0 con 0x7f8178004d40 2024-05-27T00:40:12.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.664+0000 7f817effd640 1 -- 172.21.15.111:0/3140809378 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f8180059a60 con 0x7f8178004d40 2024-05-27T00:40:12.665 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:12.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f817006bfd0 msgr2=0x7f817006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f817006bfd0 0x7f817006e490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8190141bf0 tx=0x7f8184008040 comp rx=0 tx=0).stop 2024-05-27T00:40:12.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 msgr2=0x7f819013f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f819013f9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f818000fb90 tx=0x7f818000d520 comp rx=0 tx=0).stop 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 shutdown_connections 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f817006bfd0 0x7f817006e490 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f819013e9e0 0x7f8190157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8190153c80 0x7f819013e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 --2- 172.21.15.111:0/3140809378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8178004d40 0x7f819013f9c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 >> 172.21.15.111:0/3140809378 conn(0x7f81900b1c00 msgr2=0x7f81900726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 shutdown_connections 2024-05-27T00:40:12.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.666+0000 7f8194fdc640 1 -- 172.21.15.111:0/3140809378 wait complete. 2024-05-27T00:40:12.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:12.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 Processor -- start 2024-05-27T00:40:12.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 -- start start 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 0x7fbe440b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbe44162560 0x7fbe440d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe4406cc70 con 0x7fbe440b77e0 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbe440d1e50 con 0x7fbe44162560 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.781+0000 7fbe4a2a5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbe440d2130 con 0x7fbe440b3630 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe437fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbe440b77e0 0x7fbe44161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36026/0 (socket says 172.21.15.111:36026) 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe437fe640 1 -- 172.21.15.111:0/3573415806 learned_addr learned my addr 172.21.15.111:0/3573415806 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:12.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2175170853 0 0) 0x7fbe4406cc70 con 0x7fbe440b77e0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe43fff640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 0x7fbe440b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe2c003520 con 0x7fbe440b77e0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe48842640 1 --2- 172.21.15.111:0/3573415806 >> v2:172.21.15.153:3300/0 conn(0x7fbe44162560 0x7fbe440d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3732721113 0 0) 0x7fbe2c003520 con 0x7fbe440b77e0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 msgr2=0x7fbe440b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 0x7fbe440b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 >> v2:172.21.15.153:3300/0 conn(0x7fbe44162560 msgr2=0x7fbe440d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 --2- 172.21.15.111:0/3573415806 >> v2:172.21.15.153:3300/0 conn(0x7fbe44162560 0x7fbe440d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe440d2410 con 0x7fbe440b77e0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fbe38001670 con 0x7fbe440b77e0 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 >> v1:172.21.15.111:6789/0 conn(0x7fbe440b77e0 legacy=0x7fbe44161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 0x7fbe2c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c003f30 0x7fbe2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe2c004d40 0x7fbe2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe440d1e50 con 0x7fbe2c004d40 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe440d2130 con 0x7fbe2c003f30 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c00b9c0 con 0x7fbe440b3630 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe48842640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe2c004d40 0x7fbe2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe43fff640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 0x7fbe2c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.782+0000 7fbe437fe640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c003f30 0x7fbe2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe48842640 1 -- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 msgr2=0x7fbe2c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe48842640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe440b3630 0x7fbe2c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe48842640 1 -- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c003f30 msgr2=0x7fbe2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe48842640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe2c003f30 0x7fbe2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe48842640 1 -- 172.21.15.111:0/3573415806 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c003520 con 0x7fbe2c004d40 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe48842640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe2c004d40 0x7fbe2c0051c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009f90 tx=0x7fbe2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a797eaf72f29097f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c009c10 con 0x7fbe2c004d40 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3c036070 con 0x7fbe2c004d40 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.783+0000 7fbe42ffd640 1 -- 172.21.15.111:0/3573415806 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c031670 con 0x7fbe2c004d40 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/3573415806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe2c004d40 msgr2=0x7fbe2c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/3573415806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe2c004d40 0x7fbe2c0051c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009f90 tx=0x7fbe2c005830 comp rx=0 tx=0).stop 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe48842640 1 -- 172.21.15.111:0/3573415806 reap_dead start 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/3573415806 shutdown_connections 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/3573415806 >> 172.21.15.111:0/3573415806 conn(0x7fbe440d42c0 msgr2=0x7fbe440d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/3573415806 shutdown_connections 2024-05-27T00:40:12.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.784+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/3573415806 wait complete. 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.785+0000 7fbe4a2a5640 1 Processor -- start 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.785+0000 7fbe4a2a5640 1 -- start start 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe4a2a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe441f11d0 0x7fbe441f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe4a2a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe441f15b0 0x7fbe440ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe4a2a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 0x7fbe441f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe4a2a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe440b7420 con 0x7fbe440cd220 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe4a2a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbe440b72a0 con 0x7fbe441f15b0 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe4a2a5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbe440b75a0 con 0x7fbe441f11d0 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe48842640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 0x7fbe441f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe48842640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 0x7fbe441f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36340/0 (socket says 172.21.15.111:36340) 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.786+0000 7fbe48842640 1 -- 172.21.15.111:0/1827465296 learned_addr learned my addr 172.21.15.111:0/1827465296 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe43fff640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe441f11d0 0x7fbe441f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe437fe640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe441f15b0 0x7fbe440ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe48842640 1 -- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe441f11d0 msgr2=0x7fbe441f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe48842640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe441f11d0 0x7fbe441f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe48842640 1 -- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe441f15b0 msgr2=0x7fbe440ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe48842640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe441f15b0 0x7fbe440ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe48842640 1 -- 172.21.15.111:0/1827465296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe440cb1b0 con 0x7fbe440cd220 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe48842640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 0x7fbe441f7420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009f90 tx=0x7fbe3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c009c10 con 0x7fbe440cd220 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3c031db0 con 0x7fbe440cd220 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c003bb0 con 0x7fbe440cd220 2024-05-27T00:40:12.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe441f4dc0 con 0x7fbe440cd220 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.787+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe440cb730 con 0x7fbe440cd220 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.788+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe44068680 con 0x7fbe440cd220 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.788+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe3c005d10 con 0x7fbe440cd220 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.789+0000 7fbe40ff9640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbe3006be30 0x7fbe3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.789+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fbe3c0ac170 con 0x7fbe440cd220 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.789+0000 7fbe43fff640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbe3006be30 0x7fbe3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:12.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.790+0000 7fbe43fff640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbe3006be30 0x7fbe3006e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe340099e0 tx=0x7fbe34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:12.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.793+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbe3c049060 con 0x7fbe440cd220 2024-05-27T00:40:12.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.970+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbe440cef10 con 0x7fbe440cd220 2024-05-27T00:40:12.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.971+0000 7fbe40ff9640 1 -- 172.21.15.111:0/1827465296 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fbe3c07a700 con 0x7fbe440cd220 2024-05-27T00:40:12.972 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:12.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbe3006be30 msgr2=0x7fbe3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbe3006be30 0x7fbe3006e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe340099e0 tx=0x7fbe34008040 comp rx=0 tx=0).stop 2024-05-27T00:40:12.974 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 msgr2=0x7fbe441f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 0x7fbe441f7420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009f90 tx=0x7fbe3c009750 comp rx=0 tx=0).stop 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 shutdown_connections 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbe3006be30 0x7fbe3006e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe440cd220 0x7fbe441f7420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe441f15b0 0x7fbe440ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 --2- 172.21.15.111:0/1827465296 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbe441f11d0 0x7fbe441f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 >> 172.21.15.111:0/1827465296 conn(0x7fbe440d42c0 msgr2=0x7fbe440b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:12.975 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 shutdown_connections 2024-05-27T00:40:12.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:12.973+0000 7fbe4a2a5640 1 -- 172.21.15.111:0/1827465296 wait complete. 2024-05-27T00:40:12.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:13.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.086+0000 7fb910a1c640 1 Processor -- start 2024-05-27T00:40:13.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.086+0000 7fb910a1c640 1 -- start start 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb910a1c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c1459c0 0x7fb90c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb910a1c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb90c0cf080 0x7fb90c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb910a1c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb90c0c7700 con 0x7fb90c0d19f0 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb910a1c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb90c0c79e0 con 0x7fb90c0cf080 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb910a1c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb90c0c7cc0 con 0x7fb90c1459c0 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb90ad9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb90c0cf080 0x7fb90c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb909d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb90c0d19f0 0x7fb90c0ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36034/0 (socket says 172.21.15.111:36034) 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb909d9b640 1 -- 172.21.15.111:0/1487843067 learned_addr learned my addr 172.21.15.111:0/1487843067 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb90a59c640 1 --2- 172.21.15.111:0/1487843067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c1459c0 0x7fb90c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb90ad9d640 1 -- 172.21.15.111:0/1487843067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c1459c0 msgr2=0x7fb90c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:13.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb90ad9d640 1 --2- 172.21.15.111:0/1487843067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c1459c0 0x7fb90c0c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb90ad9d640 1 -- 172.21.15.111:0/1487843067 >> v1:172.21.15.111:6789/0 conn(0x7fb90c0d19f0 legacy=0x7fb90c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.087+0000 7fb90ad9d640 1 -- 172.21.15.111:0/1487843067 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c0c7fa0 con 0x7fb90c0cf080 2024-05-27T00:40:13.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.088+0000 7fb90ad9d640 1 --2- 172.21.15.111:0/1487843067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c0cf080 0x7fb90c0cc950 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb904009dc0 tx=0x7fb90402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e5f7c442452f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.088+0000 7fb90959a640 1 -- 172.21.15.111:0/1487843067 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904007740 con 0x7fb90c0cf080 2024-05-27T00:40:13.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.088+0000 7fb90959a640 1 -- 172.21.15.111:0/1487843067 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9040078e0 con 0x7fb90c0cf080 2024-05-27T00:40:13.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.088+0000 7fb90959a640 1 -- 172.21.15.111:0/1487843067 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904007cb0 con 0x7fb90c0cf080 2024-05-27T00:40:13.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 -- 172.21.15.111:0/1487843067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c0cf080 msgr2=0x7fb90c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 --2- 172.21.15.111:0/1487843067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c0cf080 0x7fb90c0cc950 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb904009dc0 tx=0x7fb90402a5f0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 -- 172.21.15.111:0/1487843067 shutdown_connections 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 --2- 172.21.15.111:0/1487843067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c0cf080 0x7fb90c0cc950 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 --2- 172.21.15.111:0/1487843067 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c1459c0 0x7fb90c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 -- 172.21.15.111:0/1487843067 >> 172.21.15.111:0/1487843067 conn(0x7fb90c0d1f60 msgr2=0x7fb90c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 -- 172.21.15.111:0/1487843067 shutdown_connections 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.089+0000 7fb910a1c640 1 -- 172.21.15.111:0/1487843067 wait complete. 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.090+0000 7fb910a1c640 1 Processor -- start 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.090+0000 7fb910a1c640 1 -- start start 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb910a1c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c0cf080 0x7fb90c1403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb910a1c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c0d19f0 0x7fb90c1423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb910a1c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 0x7fb90c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb910a1c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c145f20 con 0x7fb90c1459c0 2024-05-27T00:40:13.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb910a1c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c145da0 con 0x7fb90c0d19f0 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb910a1c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb90c0cce90 con 0x7fb90c0cf080 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 0x7fb90c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 0x7fb90c1428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36352/0 (socket says 172.21.15.111:36352) 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 -- 172.21.15.111:0/939968413 learned_addr learned my addr 172.21.15.111:0/939968413 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90a59c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c0cf080 0x7fb90c1403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb909d9b640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c0d19f0 0x7fb90c1423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 -- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c0cf080 msgr2=0x7fb90c1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c0cf080 0x7fb90c1403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 -- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c0d19f0 msgr2=0x7fb90c1423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c0d19f0 0x7fb90c1423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.091+0000 7fb90ad9d640 1 -- 172.21.15.111:0/939968413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c143050 con 0x7fb90c1459c0 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.092+0000 7fb90ad9d640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 0x7fb90c1428f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb904009ee0 tx=0x7fb904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.092+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904040070 con 0x7fb90c1459c0 2024-05-27T00:40:13.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.092+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb904031bc0 con 0x7fb90c1459c0 2024-05-27T00:40:13.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.092+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c143330 con 0x7fb90c1459c0 2024-05-27T00:40:13.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.092+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904045460 con 0x7fb90c1459c0 2024-05-27T00:40:13.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.092+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c0b5390 con 0x7fb90c1459c0 2024-05-27T00:40:13.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.093+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c068680 con 0x7fb90c1459c0 2024-05-27T00:40:13.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.093+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb904039040 con 0x7fb90c1459c0 2024-05-27T00:40:13.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.094+0000 7fb8faffd640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8ec06be30 0x7fb8ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.094+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fb9040acaf0 con 0x7fb90c1459c0 2024-05-27T00:40:13.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.094+0000 7fb90a59c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8ec06be30 0x7fb8ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.094+0000 7fb90a59c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8ec06be30 0x7fb8ec06e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0079a0 tx=0x7fb8fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.098+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb90407aab0 con 0x7fb90c1459c0 2024-05-27T00:40:13.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.269+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb90c070a20 con 0x7fb90c1459c0 2024-05-27T00:40:13.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.270+0000 7fb8faffd640 1 -- 172.21.15.111:0/939968413 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb90407edb0 con 0x7fb90c1459c0 2024-05-27T00:40:13.271 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8ec06be30 msgr2=0x7fb8ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8ec06be30 0x7fb8ec06e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0079a0 tx=0x7fb8fc008040 comp rx=0 tx=0).stop 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 msgr2=0x7fb90c1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 0x7fb90c1428f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb904009ee0 tx=0x7fb904004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 shutdown_connections 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8ec06be30 0x7fb8ec06e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb90c1459c0 0x7fb90c1428f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb90c0d19f0 0x7fb90c1423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 --2- 172.21.15.111:0/939968413 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb90c0cf080 0x7fb90c1403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 >> 172.21.15.111:0/939968413 conn(0x7fb90c0d1f60 msgr2=0x7fb90c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:13.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 shutdown_connections 2024-05-27T00:40:13.274 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.272+0000 7fb910a1c640 1 -- 172.21.15.111:0/939968413 wait complete. 2024-05-27T00:40:13.285 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T00:40:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.385+0000 7ff994cbd640 1 Processor -- start 2024-05-27T00:40:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.385+0000 7ff994cbd640 1 -- start start 2024-05-27T00:40:13.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff994cbd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff99006b390 0x7ff9900cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff994cbd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff9900726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff994cbd640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9900b36e0 con 0x7ff990144000 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff994cbd640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff9900b42f0 con 0x7ff99006b390 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff994cbd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900b39c0 con 0x7ff9900cdc80 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98e59c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff990144000 0x7ff9900b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36050/0 (socket says 172.21.15.111:36050) 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98e59c640 1 -- 172.21.15.111:0/482320144 learned_addr learned my addr 172.21.15.111:0/482320144 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff985d9b640 1 --2- 172.21.15.111:0/482320144 >> v2:172.21.15.153:3300/0 conn(0x7ff99006b390 0x7ff9900cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff9900726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034353923 0 0) 0x7ff9900b36e0 con 0x7ff990144000 2024-05-27T00:40:13.390 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff97c003520 con 0x7ff990144000 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2356959724 0 0) 0x7ff97c003520 con 0x7ff990144000 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 msgr2=0x7ff9900726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff9900726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 >> v2:172.21.15.153:3300/0 conn(0x7ff99006b390 msgr2=0x7ff9900cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 --2- 172.21.15.111:0/482320144 >> v2:172.21.15.153:3300/0 conn(0x7ff99006b390 0x7ff9900cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9900b3ca0 con 0x7ff990144000 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7ff980001670 con 0x7ff990144000 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff985d9b640 1 --2- 172.21.15.111:0/482320144 >> v2:172.21.15.153:3300/0 conn(0x7ff99006b390 0x7ff9900cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 >> v1:172.21.15.111:6789/0 conn(0x7ff990144000 legacy=0x7ff9900b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97c0039b0 0x7ff97c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff97c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff97c004d40 0x7ff97c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900b42f0 con 0x7ff97c004d40 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900b39c0 con 0x7ff97c0039b0 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.386+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c00b9c0 con 0x7ff9900cdc80 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff97c004d40 0x7ff97c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98e59c640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97c0039b0 0x7ff97c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff985d9b640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff97c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 -- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 msgr2=0x7ff97c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff97c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 -- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97c0039b0 msgr2=0x7ff97c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff97c0039b0 0x7ff97c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 -- 172.21.15.111:0/482320144 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c003520 con 0x7ff97c004d40 2024-05-27T00:40:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff985d9b640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff9900cdc80 0x7ff97c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.387+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff97c004d40 0x7ff97c0051c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff988009f90 tx=0x7ff97c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7227fe81772cf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988009c10 con 0x7ff97c004d40 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff988036070 con 0x7ff97c004d40 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff98dd9b640 1 -- 172.21.15.111:0/482320144 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988031810 con 0x7ff97c004d40 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff994cbd640 1 -- 172.21.15.111:0/482320144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff97c004d40 msgr2=0x7ff97c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff994cbd640 1 --2- 172.21.15.111:0/482320144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff97c004d40 0x7ff97c0051c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff988009f90 tx=0x7ff97c005830 comp rx=0 tx=0).stop 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff98ed9d640 1 -- 172.21.15.111:0/482320144 reap_dead start 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff994cbd640 1 -- 172.21.15.111:0/482320144 shutdown_connections 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff994cbd640 1 -- 172.21.15.111:0/482320144 >> 172.21.15.111:0/482320144 conn(0x7ff9900b1c00 msgr2=0x7ff9900b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff994cbd640 1 -- 172.21.15.111:0/482320144 shutdown_connections 2024-05-27T00:40:13.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.388+0000 7ff994cbd640 1 -- 172.21.15.111:0/482320144 wait complete. 2024-05-27T00:40:13.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 Processor -- start 2024-05-27T00:40:13.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 -- start start 2024-05-27T00:40:13.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97c004d40 0x7ff9900aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 0x7ff99015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9900aa410 0x7ff9900ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff990072850 con 0x7ff99013f1e0 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900726d0 con 0x7ff9900aa410 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff994cbd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900729d0 con 0x7ff97c004d40 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.390+0000 7ff985d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 0x7ff99015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 0x7ff99015a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36380/0 (socket says 172.21.15.111:36380) 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 -- 172.21.15.111:0/2640976068 learned_addr learned my addr 172.21.15.111:0/2640976068 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff98e59c640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97c004d40 0x7ff9900aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9900aa410 0x7ff9900ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 -- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97c004d40 msgr2=0x7ff9900aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97c004d40 0x7ff9900aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 -- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9900aa410 msgr2=0x7ff9900ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9900aa410 0x7ff9900ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 -- 172.21.15.111:0/2640976068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9900af240 con 0x7ff99013f1e0 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff98e59c640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97c004d40 0x7ff9900aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:13.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff98ed9d640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9900aa410 0x7ff9900ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff985d9b640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 0x7ff99015a850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff9780118d0 tx=0x7ff978011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97800fbd0 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97800fd70 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9900a9650 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9780156a0 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.391+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9900a9c10 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.392+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff990068680 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.393+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff978015840 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.393+0000 7ff9877fe640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff97006be30 0x7ff97006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.393+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff97808bdb0 con 0x7ff99013f1e0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.393+0000 7ff98e59c640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff97006be30 0x7ff97006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.394+0000 7ff98e59c640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff97006be30 0x7ff97006e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff980007cd0 tx=0x7ff980009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.397+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff978019030 con 0x7ff99013f1e0 2024-05-27T00:40:13.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.581+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff990142cc0 con 0x7ff99013f1e0 2024-05-27T00:40:13.582 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.581+0000 7ff9877fe640 1 -- 172.21.15.111:0/2640976068 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7ff97805a340 con 0x7ff99013f1e0 2024-05-27T00:40:13.582 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T00:40:13.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff97006be30 msgr2=0x7ff97006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff97006be30 0x7ff97006e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff980007cd0 tx=0x7ff980009040 comp rx=0 tx=0).stop 2024-05-27T00:40:13.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 msgr2=0x7ff99015a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 0x7ff99015a850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff9780118d0 tx=0x7ff978011d90 comp rx=0 tx=0).stop 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 shutdown_connections 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff97006be30 0x7ff97006e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9900aa410 0x7ff9900ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff99013f1e0 0x7ff99015a850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 --2- 172.21.15.111:0/2640976068 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff97c004d40 0x7ff9900aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 >> 172.21.15.111:0/2640976068 conn(0x7ff9900b1c00 msgr2=0x7ff990140b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 shutdown_connections 2024-05-27T00:40:13.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.583+0000 7ff994cbd640 1 -- 172.21.15.111:0/2640976068 wait complete. 2024-05-27T00:40:13.595 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-27T00:40:13.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:13.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-27T00:40:13.621 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T00:40:13.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:13.621 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-27T00:40:13.678 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:40:13.678 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:13.697 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:40:13.698 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:13.756 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T00:40:13.756 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:13.814 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:13.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.913+0000 7f7fc8ff9640 1 Processor -- start 2024-05-27T00:40:13.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.913+0000 7f7fc8ff9640 1 -- start start 2024-05-27T00:40:13.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc8ff9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4151740 0x7f7fc4071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc8ff9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7fc406c330 0x7f7fc40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc8ff9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc4071fe0 con 0x7f7fc4071b40 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc8ff9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc40722c0 con 0x7f7fc406c330 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc8ff9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc40725a0 con 0x7f7fc4151740 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7fc406c330 0x7f7fc40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc1d9b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f7fc4071b40 0x7f7fc406bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36056/0 (socket says 172.21.15.111:36056) 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc2d9d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7fc406c330 0x7f7fc40cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36392/0 (socket says 172.21.15.111:36392) 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc1d9b640 1 -- 172.21.15.111:0/2450137521 learned_addr learned my addr 172.21.15.111:0/2450137521 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc259c640 1 --2- 172.21.15.111:0/2450137521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4151740 0x7f7fc4071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/2450137521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4151740 msgr2=0x7f7fc4071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc2d9d640 1 --2- 172.21.15.111:0/2450137521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4151740 0x7f7fc4071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/2450137521 >> v1:172.21.15.111:6789/0 conn(0x7f7fc4071b40 legacy=0x7f7fc406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/2450137521 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4072880 con 0x7f7fc406c330 2024-05-27T00:40:13.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.914+0000 7f7fc159a640 1 -- 172.21.15.111:0/2450137521 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585752922 0 0) 0x7f7fc4071fe0 con 0x7f7fc4071b40 2024-05-27T00:40:13.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.915+0000 7f7fc2d9d640 1 --2- 172.21.15.111:0/2450137521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc406c330 0x7f7fc40cf030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009d90 tx=0x7f7fbc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def241f569cb26a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.915+0000 7f7fc159a640 1 -- 172.21.15.111:0/2450137521 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc007740 con 0x7f7fc406c330 2024-05-27T00:40:13.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.915+0000 7f7fc159a640 1 -- 172.21.15.111:0/2450137521 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0078e0 con 0x7f7fc406c330 2024-05-27T00:40:13.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.915+0000 7f7fc159a640 1 -- 172.21.15.111:0/2450137521 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc007cb0 con 0x7f7fc406c330 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.915+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/2450137521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc406c330 msgr2=0x7f7fc40cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.915+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/2450137521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc406c330 0x7f7fc40cf030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009d90 tx=0x7f7fbc02a660 comp rx=0 tx=0).stop 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.916+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/2450137521 shutdown_connections 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.916+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/2450137521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc406c330 0x7f7fc40cf030 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.916+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/2450137521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4151740 0x7f7fc4071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.916+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/2450137521 >> 172.21.15.111:0/2450137521 conn(0x7f7fc40b1c00 msgr2=0x7f7fc40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.916+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/2450137521 shutdown_connections 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.916+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/2450137521 wait complete. 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.917+0000 7f7fc8ff9640 1 Processor -- start 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.917+0000 7f7fc8ff9640 1 -- start start 2024-05-27T00:40:13.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fc406c330 0x7f7fc413b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4071b40 0x7f7fc413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 0x7f7fc4145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc415b090 con 0x7f7fc4151740 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc415af10 con 0x7f7fc406c330 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc406b390 con 0x7f7fc4071b40 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 0x7f7fc4145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 0x7f7fc4145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36406/0 (socket says 172.21.15.111:36406) 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/1001015885 learned_addr learned my addr 172.21.15.111:0/1001015885 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc1d9b640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4071b40 0x7f7fc413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc259c640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fc406c330 0x7f7fc413b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4071b40 msgr2=0x7f7fc413b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4071b40 0x7f7fc413b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fc406c330 msgr2=0x7f7fc413b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fc406c330 0x7f7fc413b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 -- 172.21.15.111:0/1001015885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc413c310 con 0x7f7fc4151740 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc2d9d640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 0x7f7fc4145c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009eb0 tx=0x7f7fbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc040070 con 0x7f7fc4151740 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc031c40 con 0x7f7fc4151740 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc413c5f0 con 0x7f7fc4151740 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc009450 con 0x7f7fc4151740 2024-05-27T00:40:13.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.918+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc413cbb0 con 0x7f7fc4151740 2024-05-27T00:40:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.919+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc4068680 con 0x7f7fc4151740 2024-05-27T00:40:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.920+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fbc039040 con 0x7f7fc4151740 2024-05-27T00:40:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.920+0000 7f7fb2ffd640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7fa406be30 0x7f7fa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.920+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f7fbc0ac190 con 0x7f7fc4151740 2024-05-27T00:40:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.920+0000 7f7fc259c640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7fa406be30 0x7f7fa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:13.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.921+0000 7f7fc259c640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7fa406be30 0x7f7fa406e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4006fd0 tx=0x7f7fb4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:13.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:13.924+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7fbc07a720 con 0x7f7fc4151740 2024-05-27T00:40:14.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.102+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fc40b4520 con 0x7f7fc4151740 2024-05-27T00:40:14.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.103+0000 7f7fb2ffd640 1 -- 172.21.15.111:0/1001015885 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f7fbc07a020 con 0x7f7fc4151740 2024-05-27T00:40:14.104 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:14.104 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7fa406be30 msgr2=0x7f7fa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7fa406be30 0x7f7fa406e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4006fd0 tx=0x7f7fb4008040 comp rx=0 tx=0).stop 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 msgr2=0x7f7fc4145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 0x7f7fc4145c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009eb0 tx=0x7f7fbc004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 shutdown_connections 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f7fa406be30 0x7f7fa406e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4151740 0x7f7fc4145c20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f7fc4071b40 0x7f7fc413b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 --2- 172.21.15.111:0/1001015885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fc406c330 0x7f7fc413b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 >> 172.21.15.111:0/1001015885 conn(0x7f7fc40b1c00 msgr2=0x7f7fc40b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:14.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 shutdown_connections 2024-05-27T00:40:14.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.105+0000 7f7fc8ff9640 1 -- 172.21.15.111:0/1001015885 wait complete. 2024-05-27T00:40:14.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:14.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:14.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.219+0000 7ff6e3afc640 1 Processor -- start 2024-05-27T00:40:14.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.219+0000 7ff6e3afc640 1 -- start start 2024-05-27T00:40:14.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e3afc640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6dc144000 0x7ff6dc0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e3afc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 0x7ff6dc0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e3afc640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6dc0b3440 con 0x7ff6dc0cdc80 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e3afc640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7ff6dc0b42f0 con 0x7ff6dc144000 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e3afc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc0b36e0 con 0x7ff6dc06b390 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1097640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 0x7ff6dc0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1898640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6dc144000 0x7ff6dc0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e2099640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff6dc0cdc80 0x7ff6dc0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:40628/0 (socket says 172.21.15.111:40628) 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1097640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 0x7ff6dc0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:32878/0 (socket says 172.21.15.111:32878) 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e2099640 1 -- 172.21.15.111:0/629798345 learned_addr learned my addr 172.21.15.111:0/629798345 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1097640 1 -- 172.21.15.111:0/629798345 >> v2:172.21.15.153:3300/0 conn(0x7ff6dc144000 msgr2=0x7ff6dc0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1097640 1 --2- 172.21.15.111:0/629798345 >> v2:172.21.15.153:3300/0 conn(0x7ff6dc144000 0x7ff6dc0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1097640 1 -- 172.21.15.111:0/629798345 >> v1:172.21.15.153:6789/0 conn(0x7ff6dc0cdc80 legacy=0x7ff6dc0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.220+0000 7ff6e1097640 1 -- 172.21.15.111:0/629798345 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc0b39c0 con 0x7ff6dc06b390 2024-05-27T00:40:14.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.221+0000 7ff6e1898640 1 --2- 172.21.15.111:0/629798345 >> v2:172.21.15.153:3300/0 conn(0x7ff6dc144000 0x7ff6dc0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.221+0000 7ff6e1097640 1 --2- 172.21.15.111:0/629798345 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 0x7ff6dc0cd740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097a0 tx=0x7ff6cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b9e9b694574166 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.221+0000 7ff6e0896640 1 -- 172.21.15.111:0/629798345 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc037070 con 0x7ff6dc06b390 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.221+0000 7ff6e0896640 1 -- 172.21.15.111:0/629798345 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6cc032440 con 0x7ff6dc06b390 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.221+0000 7ff6e0896640 1 -- 172.21.15.111:0/629798345 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc032810 con 0x7ff6dc06b390 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 -- 172.21.15.111:0/629798345 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 msgr2=0x7ff6dc0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/629798345 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 0x7ff6dc0cd740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097a0 tx=0x7ff6cc02a180 comp rx=0 tx=0).stop 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 -- 172.21.15.111:0/629798345 shutdown_connections 2024-05-27T00:40:14.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/629798345 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc06b390 0x7ff6dc0cd740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/629798345 >> v2:172.21.15.153:3300/0 conn(0x7ff6dc144000 0x7ff6dc0b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 -- 172.21.15.111:0/629798345 >> 172.21.15.111:0/629798345 conn(0x7ff6dc0b1c00 msgr2=0x7ff6dc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:14.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 -- 172.21.15.111:0/629798345 shutdown_connections 2024-05-27T00:40:14.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.222+0000 7ff6e3afc640 1 -- 172.21.15.111:0/629798345 wait complete. 2024-05-27T00:40:14.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.223+0000 7ff6e3afc640 1 Processor -- start 2024-05-27T00:40:14.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.223+0000 7ff6e3afc640 1 -- start start 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e3afc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 0x7ff6dc13c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e3afc640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6dc0cdc80 0x7ff6dc1e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e3afc640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc144000 0x7ff6dc142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e3afc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc072de0 con 0x7ff6dc06b390 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e3afc640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc0b3f80 con 0x7ff6dc0cdc80 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e3afc640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc072f60 con 0x7ff6dc144000 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 0x7ff6dc13c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 0x7ff6dc13c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36414/0 (socket says 172.21.15.111:36414) 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 -- 172.21.15.111:0/1317689716 learned_addr learned my addr 172.21.15.111:0/1317689716 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e2099640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc144000 0x7ff6dc142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1097640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6dc0cdc80 0x7ff6dc1e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 -- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc144000 msgr2=0x7ff6dc142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc144000 0x7ff6dc142390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 -- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6dc0cdc80 msgr2=0x7ff6dc1e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6dc0cdc80 0x7ff6dc1e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.224+0000 7ff6e1898640 1 -- 172.21.15.111:0/1317689716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc1e3630 con 0x7ff6dc06b390 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.225+0000 7ff6e1898640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 0x7ff6dc13c9b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800e140 tx=0x7ff6d800e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.225+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800b2d0 con 0x7ff6dc06b390 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.225+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d801a070 con 0x7ff6dc06b390 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.225+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8009740 con 0x7ff6dc06b390 2024-05-27T00:40:14.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.225+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc1e3910 con 0x7ff6dc06b390 2024-05-27T00:40:14.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.225+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc1e3ed0 con 0x7ff6dc06b390 2024-05-27T00:40:14.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.226+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc068680 con 0x7ff6dc06b390 2024-05-27T00:40:14.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.226+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6d8005d10 con 0x7ff6dc06b390 2024-05-27T00:40:14.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.227+0000 7ff6d27fc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff6c006be60 0x7ff6c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.227+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff6d808d540 con 0x7ff6dc06b390 2024-05-27T00:40:14.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.227+0000 7ff6e1097640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff6c006be60 0x7ff6c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.227+0000 7ff6e1097640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff6c006be60 0x7ff6c006e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc034000 tx=0x7ff6cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.231+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff6d8012040 con 0x7ff6dc06b390 2024-05-27T00:40:14.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.405+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6dc06c390 con 0x7ff6dc06b390 2024-05-27T00:40:14.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.405+0000 7ff6d27fc640 1 -- 172.21.15.111:0/1317689716 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7ff6d805b480 con 0x7ff6dc06b390 2024-05-27T00:40:14.406 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff6c006be60 msgr2=0x7ff6c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff6c006be60 0x7ff6c006e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc034000 tx=0x7ff6cc02e000 comp rx=0 tx=0).stop 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 msgr2=0x7ff6dc13c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 0x7ff6dc13c9b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800e140 tx=0x7ff6d800e600 comp rx=0 tx=0).stop 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 shutdown_connections 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff6c006be60 0x7ff6c006e320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff6dc144000 0x7ff6dc142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6dc0cdc80 0x7ff6dc1e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 --2- 172.21.15.111:0/1317689716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6dc06b390 0x7ff6dc13c9b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 >> 172.21.15.111:0/1317689716 conn(0x7ff6dc0b1c00 msgr2=0x7ff6dc0af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:14.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 shutdown_connections 2024-05-27T00:40:14.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.407+0000 7ff6e3afc640 1 -- 172.21.15.111:0/1317689716 wait complete. 2024-05-27T00:40:14.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.521+0000 7f1b4ce76640 1 Processor -- start 2024-05-27T00:40:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.521+0000 7f1b4ce76640 1 -- start start 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4ce76640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b48144000 0x7f1b480b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4ce76640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1b4806b390 0x7f1b480cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4ce76640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b480b3440 con 0x7f1b480cdc80 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4ce76640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1b480b42f0 con 0x7f1b48144000 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4ce76640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1b480b36e0 con 0x7f1b4806b390 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4659c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1b48144000 0x7f1b480b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b46d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1b480cdc80 0x7f1b480726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36068/0 (socket says 172.21.15.111:36068) 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b46d9d640 1 -- 172.21.15.111:0/2268726483 learned_addr learned my addr 172.21.15.111:0/2268726483 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4659c640 1 --2- 172.21.15.111:0/2268726483 >> v2:172.21.15.111:3300/0 conn(0x7f1b48144000 0x7f1b480b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36420/0 (socket says 172.21.15.111:36420) 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b3dd9b640 1 --2- 172.21.15.111:0/2268726483 >> v2:172.21.15.153:3300/0 conn(0x7f1b4806b390 0x7f1b480cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4659c640 1 -- 172.21.15.111:0/2268726483 >> v2:172.21.15.153:3300/0 conn(0x7f1b4806b390 msgr2=0x7f1b480cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4659c640 1 --2- 172.21.15.111:0/2268726483 >> v2:172.21.15.153:3300/0 conn(0x7f1b4806b390 0x7f1b480cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4659c640 1 -- 172.21.15.111:0/2268726483 >> v1:172.21.15.111:6789/0 conn(0x7f1b480cdc80 legacy=0x7f1b480726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b4659c640 1 -- 172.21.15.111:0/2268726483 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b480b39c0 con 0x7f1b48144000 2024-05-27T00:40:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.522+0000 7f1b45d9b640 1 -- 172.21.15.111:0/2268726483 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584666742 0 0) 0x7f1b480b3440 con 0x7f1b480cdc80 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4659c640 1 --2- 172.21.15.111:0/2268726483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b48144000 0x7f1b480b2f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b48072650 tx=0x7f1b3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfead41de472b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b45d9b640 1 -- 172.21.15.111:0/2268726483 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037070 con 0x7f1b48144000 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b45d9b640 1 -- 172.21.15.111:0/2268726483 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38032440 con 0x7f1b48144000 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b45d9b640 1 -- 172.21.15.111:0/2268726483 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38032810 con 0x7f1b48144000 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 -- 172.21.15.111:0/2268726483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b48144000 msgr2=0x7f1b480b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/2268726483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b48144000 0x7f1b480b2f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b48072650 tx=0x7f1b3802a1a0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 -- 172.21.15.111:0/2268726483 shutdown_connections 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/2268726483 >> v2:172.21.15.153:3300/0 conn(0x7f1b4806b390 0x7f1b480cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/2268726483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b48144000 0x7f1b480b2f00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 -- 172.21.15.111:0/2268726483 >> 172.21.15.111:0/2268726483 conn(0x7f1b480b1c00 msgr2=0x7f1b480b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 -- 172.21.15.111:0/2268726483 shutdown_connections 2024-05-27T00:40:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.523+0000 7f1b4ce76640 1 -- 172.21.15.111:0/2268726483 wait complete. 2024-05-27T00:40:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.525+0000 7f1b4ce76640 1 Processor -- start 2024-05-27T00:40:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.525+0000 7f1b4ce76640 1 -- start start 2024-05-27T00:40:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 0x7f1b48154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b480cdc80 0x7f1b481384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b48144000 0x7f1b48071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48072de0 con 0x7f1b4806b390 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1b480b3f80 con 0x7f1b480cdc80 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1b48072f60 con 0x7f1b48144000 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 0x7f1b48154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 0x7f1b48154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36434/0 (socket says 172.21.15.111:36434) 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 -- 172.21.15.111:0/1247999521 learned_addr learned my addr 172.21.15.111:0/1247999521 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b3dd9b640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b480cdc80 0x7f1b481384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b46d9d640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b48144000 0x7f1b48071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 -- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b48144000 msgr2=0x7f1b48071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b48144000 0x7f1b48071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 -- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b480cdc80 msgr2=0x7f1b481384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b480cdc80 0x7f1b481384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 -- 172.21.15.111:0/1247999521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4813b860 con 0x7f1b4806b390 2024-05-27T00:40:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b46d9d640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b48144000 0x7f1b48071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b3dd9b640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b480cdc80 0x7f1b481384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4659c640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 0x7f1b48154740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000c00 tx=0x7f1b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38037050 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38038040 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4813ace0 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38004450 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.526+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4813b2a0 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.528+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b48068680 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.528+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b3803c6d0 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.528+0000 7f1b3f7fe640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b2806be60 0x7f1b2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.529+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f1b380abbc0 con 0x7f1b4806b390 2024-05-27T00:40:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.529+0000 7f1b3dd9b640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b2806be60 0x7f1b2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.529+0000 7f1b3dd9b640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b2806be60 0x7f1b2806e320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b481f72d0 tx=0x7f1b30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.534 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.533+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b3807a1e0 con 0x7f1b4806b390 2024-05-27T00:40:14.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.705+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b4815aaf0 con 0x7f1b4806b390 2024-05-27T00:40:14.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.705+0000 7f1b3f7fe640 1 -- 172.21.15.111:0/1247999521 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f1b38079ae0 con 0x7f1b4806b390 2024-05-27T00:40:14.706 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:14.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b2806be60 msgr2=0x7f1b2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b2806be60 0x7f1b2806e320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b481f72d0 tx=0x7f1b30008040 comp rx=0 tx=0).stop 2024-05-27T00:40:14.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 msgr2=0x7f1b48154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 0x7f1b48154740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000c00 tx=0x7f1b380046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 shutdown_connections 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b2806be60 0x7f1b2806e320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1b48144000 0x7f1b48071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b480cdc80 0x7f1b481384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 --2- 172.21.15.111:0/1247999521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4806b390 0x7f1b48154740 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 >> 172.21.15.111:0/1247999521 conn(0x7f1b480b1c00 msgr2=0x7f1b481373a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 shutdown_connections 2024-05-27T00:40:14.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.707+0000 7f1b4ce76640 1 -- 172.21.15.111:0/1247999521 wait complete. 2024-05-27T00:40:14.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:14.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.821+0000 7ff478d12640 1 Processor -- start 2024-05-27T00:40:14.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.821+0000 7ff478d12640 1 -- start start 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff478d12640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 0x7ff4740b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff478d12640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4740b3440 con 0x7ff4740726d0 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff478d12640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4740b36e0 con 0x7ff47406b390 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff478d12640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff4740b42f0 con 0x7ff474144000 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff472d9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff4740726d0 0x7ff4740b4780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36082/0 (socket says 172.21.15.111:36082) 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff472d9d640 1 -- 172.21.15.111:0/2436162143 learned_addr learned my addr 172.21.15.111:0/2436162143 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff47259c640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 0x7ff4740b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.822+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331905798 0 0) 0x7ff4740b3440 con 0x7ff4740726d0 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff45c003520 con 0x7ff4740726d0 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3849071542 0 0) 0x7ff45c003520 con 0x7ff4740726d0 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 msgr2=0x7ff4740b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:14.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 0x7ff4740b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 >> v1:172.21.15.153:6789/0 conn(0x7ff47406b390 legacy=0x7ff4740cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4740b39c0 con 0x7ff4740726d0 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7ff46c001670 con 0x7ff4740726d0 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 >> v1:172.21.15.111:6789/0 conn(0x7ff4740726d0 legacy=0x7ff4740b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 0x7ff45c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff45c003f80 0x7ff45c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff45c004d40 0x7ff45c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740b42f0 con 0x7ff45c003f80 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c00bad0 con 0x7ff45c004d40 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c00dee0 con 0x7ff474144000 2024-05-27T00:40:14.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff471d9b640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff45c003f80 0x7ff45c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff47259c640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 0x7ff45c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff472d9d640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff45c004d40 0x7ff45c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff471d9b640 1 -- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 msgr2=0x7ff45c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff471d9b640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff474144000 0x7ff45c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff471d9b640 1 -- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff45c004d40 msgr2=0x7ff45c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff471d9b640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff45c004d40 0x7ff45c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.823+0000 7ff471d9b640 1 -- 172.21.15.111:0/2436162143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c003520 con 0x7ff45c003f80 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.824+0000 7ff471d9b640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff45c003f80 0x7ff45c004800 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff47406b840 tx=0x7ff45c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7d2b35074e01a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.824+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468001370 con 0x7ff45c003f80 2024-05-27T00:40:14.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.824+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4680052d0 con 0x7ff45c003f80 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.824+0000 7ff47159a640 1 -- 172.21.15.111:0/2436162143 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46802ab90 con 0x7ff45c003f80 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff478d12640 1 -- 172.21.15.111:0/2436162143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff45c003f80 msgr2=0x7ff45c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff478d12640 1 --2- 172.21.15.111:0/2436162143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff45c003f80 0x7ff45c004800 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff47406b840 tx=0x7ff45c005830 comp rx=0 tx=0).stop 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff472d9d640 1 -- 172.21.15.111:0/2436162143 reap_dead start 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff478d12640 1 -- 172.21.15.111:0/2436162143 shutdown_connections 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff478d12640 1 -- 172.21.15.111:0/2436162143 >> 172.21.15.111:0/2436162143 conn(0x7ff4740b1c00 msgr2=0x7ff4740b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff478d12640 1 -- 172.21.15.111:0/2436162143 shutdown_connections 2024-05-27T00:40:14.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.825+0000 7ff478d12640 1 -- 172.21.15.111:0/2436162143 wait complete. 2024-05-27T00:40:14.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.826+0000 7ff478d12640 1 Processor -- start 2024-05-27T00:40:14.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.826+0000 7ff478d12640 1 -- start start 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff478d12640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff45c003f80 0x7ff47414cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff478d12640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 0x7ff47413fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff478d12640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47413ff40 0x7ff4741f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff478d12640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740cde50 con 0x7ff4741ef850 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff478d12640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740b3f80 con 0x7ff47413ff40 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff478d12640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff4740b2f00 con 0x7ff45c003f80 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 0x7ff47413fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 0x7ff47413fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36454/0 (socket says 172.21.15.111:36454) 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 -- 172.21.15.111:0/3107893478 learned_addr learned my addr 172.21.15.111:0/3107893478 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff472d9d640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47413ff40 0x7ff4741f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 -- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff45c003f80 msgr2=0x7ff47414cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff45c003f80 0x7ff47414cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 -- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47413ff40 msgr2=0x7ff4741f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47413ff40 0x7ff4741f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 -- 172.21.15.111:0/3107893478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47413d640 con 0x7ff4741ef850 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.827+0000 7ff471d9b640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 0x7ff47413fa00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff46802a540 tx=0x7ff468001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.828+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468037040 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.828+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff468040070 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.828+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff474140530 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.828+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468001370 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.828+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff474141870 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.828+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff474141380 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.829+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4680052d0 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.829+0000 7ff462ffd640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff45406be30 0x7ff45406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.830+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff4680aaf30 con 0x7ff4741ef850 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.830+0000 7ff47259c640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff45406be30 0x7ff45406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:14.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.830+0000 7ff47259c640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff45406be30 0x7ff45406e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff464009fc0 tx=0x7ff464009750 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:14.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:14.834+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff468079430 con 0x7ff4741ef850 2024-05-27T00:40:15.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.006+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff474068680 con 0x7ff4741ef850 2024-05-27T00:40:15.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.006+0000 7ff462ffd640 1 -- 172.21.15.111:0/3107893478 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ff468078d30 con 0x7ff4741ef850 2024-05-27T00:40:15.007 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff45406be30 msgr2=0x7ff45406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff45406be30 0x7ff45406e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff464009fc0 tx=0x7ff464009750 comp rx=0 tx=0).stop 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 msgr2=0x7ff47413fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 0x7ff47413fa00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff46802a540 tx=0x7ff468001120 comp rx=0 tx=0).stop 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 shutdown_connections 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff45406be30 0x7ff45406e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47413ff40 0x7ff4741f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4741ef850 0x7ff47413fa00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 --2- 172.21.15.111:0/3107893478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff45c003f80 0x7ff47414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 >> 172.21.15.111:0/3107893478 conn(0x7ff4740b1c00 msgr2=0x7ff474072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:15.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 shutdown_connections 2024-05-27T00:40:15.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.008+0000 7ff478d12640 1 -- 172.21.15.111:0/3107893478 wait complete. 2024-05-27T00:40:15.020 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:15.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.123+0000 7f733bfb6640 1 Processor -- start 2024-05-27T00:40:15.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.123+0000 7f733bfb6640 1 -- start start 2024-05-27T00:40:15.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f733bfb6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 0x7f733406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f733bfb6640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7334071d10 con 0x7f733406bc20 2024-05-27T00:40:15.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f733bfb6640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7334071fe0 con 0x7f7334151740 2024-05-27T00:40:15.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f733bfb6640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f73340722c0 con 0x7f73340b42f0 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7339551640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f733406bc20 0x7f73340cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36088/0 (socket says 172.21.15.111:36088) 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7339551640 1 -- 172.21.15.111:0/1651523292 learned_addr learned my addr 172.21.15.111:0/1651523292 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117131312 0 0) 0x7f7334071d10 con 0x7f733406bc20 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7320003520 con 0x7f733406bc20 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f733a553640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 0x7f733406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2036338582 0 0) 0x7f7320003520 con 0x7f733406bc20 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 msgr2=0x7f733406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 0x7f733406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 >> v1:172.21.15.153:6789/0 conn(0x7f7334151740 legacy=0x7f7334071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73340725a0 con 0x7f733406bc20 2024-05-27T00:40:15.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f7324001670 con 0x7f733406bc20 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 >> v1:172.21.15.111:6789/0 conn(0x7f733406bc20 legacy=0x7f73340cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200039e0 0x7f7320003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 0x7f7320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7320004d40 0x7f73200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73340722c0 con 0x7f73200039e0 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f732000bb50 con 0x7f7320004d40 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.124+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f732000bd20 con 0x7f73340b42f0 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200039e0 0x7f7320003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339551640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 0x7f7320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f733a553640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7320004d40 0x7f73200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 -- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 msgr2=0x7f7320004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73340b42f0 0x7f7320004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 -- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7320004d40 msgr2=0x7f73200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7320004d40 0x7f73200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 -- 172.21.15.111:0/1651523292 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7320003520 con 0x7f73200039e0 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f733a553640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7320004d40 0x7f73200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.125+0000 7f7339d52640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200039e0 0x7f7320003e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7330001780 tx=0x7f7320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751c2bdfc49a0f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330001370 con 0x7f73200039e0 2024-05-27T00:40:15.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73300052d0 con 0x7f73200039e0 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f7338d50640 1 -- 172.21.15.111:0/1651523292 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733002ac00 con 0x7f73200039e0 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733bfb6640 1 -- 172.21.15.111:0/1651523292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200039e0 msgr2=0x7f7320003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1651523292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73200039e0 0x7f7320003e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7330001780 tx=0x7f7320005830 comp rx=0 tx=0).stop 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733a553640 1 -- 172.21.15.111:0/1651523292 reap_dead start 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733bfb6640 1 -- 172.21.15.111:0/1651523292 shutdown_connections 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733bfb6640 1 -- 172.21.15.111:0/1651523292 >> 172.21.15.111:0/1651523292 conn(0x7f73340b1c00 msgr2=0x7f73340b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733bfb6640 1 -- 172.21.15.111:0/1651523292 shutdown_connections 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.126+0000 7f733bfb6640 1 -- 172.21.15.111:0/1651523292 wait complete. 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.127+0000 7f733bfb6640 1 Processor -- start 2024-05-27T00:40:15.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.127+0000 7f733bfb6640 1 -- start start 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f733bfb6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73200039e0 0x7f733412ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f733bfb6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 0x7f733415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f733bfb6640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7334149580 0x7f7334149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f733bfb6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f733415af10 con 0x7f733415cef0 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f733bfb6640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f7334151b20 con 0x7f7334149580 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f733bfb6640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f733415b090 con 0x7f73200039e0 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 0x7f733415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 0x7f733415d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36476/0 (socket says 172.21.15.111:36476) 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 -- 172.21.15.111:0/1257968938 learned_addr learned my addr 172.21.15.111:0/1257968938 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 -- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73200039e0 msgr2=0x7f733412ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73200039e0 0x7f733412ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 -- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7334149580 msgr2=0x7f7334149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7334149580 0x7f7334149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.128+0000 7f7339551640 1 -- 172.21.15.111:0/1257968938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733412f380 con 0x7f733415cef0 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.129+0000 7f7339551640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 0x7f733415d350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f732400e7b0 tx=0x7f732400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.129+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732400c8d0 con 0x7f733415cef0 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.129+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7324018070 con 0x7f733415cef0 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.129+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7334143d40 con 0x7f733415cef0 2024-05-27T00:40:15.136 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.129+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732400ccd0 con 0x7f733415cef0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.129+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7334144300 con 0x7f733415cef0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.130+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733415d890 con 0x7f733415cef0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.130+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7324013420 con 0x7f733415cef0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.131+0000 7f732a7fc640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f731806be60 0x7f731806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.131+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f732408ade0 con 0x7f733415cef0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.131+0000 7f7339d52640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f731806be60 0x7f731806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.132+0000 7f7339d52640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f731806be60 0x7f731806e320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7330032000 tx=0x7f733002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.135+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f733412f380 con 0x7f733415cef0 2024-05-27T00:40:15.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.307+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7334068680 con 0x7f733415cef0 2024-05-27T00:40:15.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.307+0000 7f732a7fc640 1 -- 172.21.15.111:0/1257968938 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f7324059400 con 0x7f733415cef0 2024-05-27T00:40:15.308 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f731806be60 msgr2=0x7f731806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f731806be60 0x7f731806e320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7330032000 tx=0x7f733002c000 comp rx=0 tx=0).stop 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 msgr2=0x7f733415d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 0x7f733415d350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f732400e7b0 tx=0x7f732400ec70 comp rx=0 tx=0).stop 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 shutdown_connections 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f731806be60 0x7f731806e320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7334149580 0x7f7334149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f733415cef0 0x7f733415d350 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 --2- 172.21.15.111:0/1257968938 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73200039e0 0x7f733412ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 >> 172.21.15.111:0/1257968938 conn(0x7f73340b1c00 msgr2=0x7f7334159260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:15.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 shutdown_connections 2024-05-27T00:40:15.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.309+0000 7f733bfb6640 1 -- 172.21.15.111:0/1257968938 wait complete. 2024-05-27T00:40:15.321 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T00:40:15.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.421+0000 7fbd6a438640 1 Processor -- start 2024-05-27T00:40:15.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.421+0000 7fbd6a438640 1 -- start start 2024-05-27T00:40:15.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd6a438640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd64151740 0x7fbd64071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd6a438640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd64071b40 0x7fbd6406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd6a438640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbd6406c160 0x7fbd640cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd6a438640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fbd6406c5d0 con 0x7fbd64151740 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd6a438640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fbd64071fe0 con 0x7fbd6406c160 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd6a438640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbd640722c0 con 0x7fbd64071b40 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd64151740 0x7fbd64071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd64151740 0x7fbd64071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36486/0 (socket says 172.21.15.111:36486) 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 -- 172.21.15.111:0/2306187668 learned_addr learned my addr 172.21.15.111:0/2306187668 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd689d5640 1 --2- 172.21.15.111:0/2306187668 >> v2:172.21.15.153:3300/0 conn(0x7fbd6406c160 0x7fbd640cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 -- 172.21.15.111:0/2306187668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd64071b40 msgr2=0x7fbd6406bc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 --2- 172.21.15.111:0/2306187668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd64071b40 0x7fbd6406bc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 -- 172.21.15.111:0/2306187668 >> v2:172.21.15.153:3300/0 conn(0x7fbd6406c160 msgr2=0x7fbd640cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 --2- 172.21.15.111:0/2306187668 >> v2:172.21.15.153:3300/0 conn(0x7fbd6406c160 0x7fbd640cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.422+0000 7fbd63fff640 1 -- 172.21.15.111:0/2306187668 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd640725a0 con 0x7fbd64151740 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.423+0000 7fbd63fff640 1 --2- 172.21.15.111:0/2306187668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd64071600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbd6406f490 tx=0x7fbd5402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b328a3a1981bca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.423+0000 7fbd637fe640 1 -- 172.21.15.111:0/2306187668 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54037070 con 0x7fbd64151740 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.423+0000 7fbd637fe640 1 -- 172.21.15.111:0/2306187668 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd54032440 con 0x7fbd64151740 2024-05-27T00:40:15.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.423+0000 7fbd637fe640 1 -- 172.21.15.111:0/2306187668 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54032810 con 0x7fbd64151740 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 -- 172.21.15.111:0/2306187668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 msgr2=0x7fbd64071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 --2- 172.21.15.111:0/2306187668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd64071600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbd6406f490 tx=0x7fbd5402a200 comp rx=0 tx=0).stop 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 -- 172.21.15.111:0/2306187668 shutdown_connections 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 --2- 172.21.15.111:0/2306187668 >> v2:172.21.15.153:3300/0 conn(0x7fbd6406c160 0x7fbd640cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 --2- 172.21.15.111:0/2306187668 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd64071b40 0x7fbd6406bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 --2- 172.21.15.111:0/2306187668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd64071600 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 -- 172.21.15.111:0/2306187668 >> 172.21.15.111:0/2306187668 conn(0x7fbd640b1c00 msgr2=0x7fbd640b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 -- 172.21.15.111:0/2306187668 shutdown_connections 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.424+0000 7fbd6a438640 1 -- 172.21.15.111:0/2306187668 wait complete. 2024-05-27T00:40:15.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.425+0000 7fbd6a438640 1 Processor -- start 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.425+0000 7fbd6a438640 1 -- start start 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd6a438640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd6406c160 0x7fbd640b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd6a438640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd64071b40 0x7fbd6414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd6a438640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd6414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd6a438640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6415af10 con 0x7fbd64151740 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd6a438640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64151b20 con 0x7fbd64071b40 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd6a438640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6415b090 con 0x7fbd6406c160 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd6414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd6414fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36492/0 (socket says 172.21.15.111:36492) 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 -- 172.21.15.111:0/1261158070 learned_addr learned my addr 172.21.15.111:0/1261158070 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd63fff640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd6406c160 0x7fbd640b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd5bfff640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd64071b40 0x7fbd6414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 -- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd6406c160 msgr2=0x7fbd640b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd6406c160 0x7fbd640b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 -- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd64071b40 msgr2=0x7fbd6414f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd64071b40 0x7fbd6414f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 -- 172.21.15.111:0/1261158070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd641f5560 con 0x7fbd64151740 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd5bfff640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd64071b40 0x7fbd6414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd63fff640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd6406c160 0x7fbd640b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.426+0000 7fbd689d5640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd6414fad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c007e40 tx=0x7fbd5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.427+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c014620 con 0x7fbd64151740 2024-05-27T00:40:15.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.427+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5c019070 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.427+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd641f5840 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.427+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c014a20 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.427+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641f5e00 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.428+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd64068680 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.428+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd5c014bc0 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.428+0000 7fbd617fa640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbd4406be30 0x7fbd4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.429+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fbd5c08bae0 con 0x7fbd64151740 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.429+0000 7fbd63fff640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbd4406be30 0x7fbd4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.429+0000 7fbd63fff640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbd4406be30 0x7fbd4406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd54034000 tx=0x7fbd5402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.433+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbd5c05a100 con 0x7fbd64151740 2024-05-27T00:40:15.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.618+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd640b4a50 con 0x7fbd64151740 2024-05-27T00:40:15.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.619+0000 7fbd617fa640 1 -- 172.21.15.111:0/1261158070 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fbd5c059a00 con 0x7fbd64151740 2024-05-27T00:40:15.620 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbd4406be30 msgr2=0x7fbd4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbd4406be30 0x7fbd4406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd54034000 tx=0x7fbd5402e000 comp rx=0 tx=0).stop 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 msgr2=0x7fbd6414fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd6414fad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c007e40 tx=0x7fbd5c00e4b0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 shutdown_connections 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fbd4406be30 0x7fbd4406e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd64151740 0x7fbd6414fad0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.622 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd64071b40 0x7fbd6414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 --2- 172.21.15.111:0/1261158070 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fbd6406c160 0x7fbd640b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 >> 172.21.15.111:0/1261158070 conn(0x7fbd640b1c00 msgr2=0x7fbd641506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:15.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 shutdown_connections 2024-05-27T00:40:15.623 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.621+0000 7fbd6a438640 1 -- 172.21.15.111:0/1261158070 wait complete. 2024-05-27T00:40:15.633 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-27T00:40:15.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:15.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-27T00:40:15.665 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T00:40:15.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:15.666 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-27T00:40:15.724 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-27T00:40:15.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:40:15.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.823+0000 7f04c9d34640 1 Processor -- start 2024-05-27T00:40:15.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.823+0000 7f04c9d34640 1 -- start start 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c9d34640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04c40cf080 0x7f04c4166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c9d34640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c40c7700 con 0x7f04c41459c0 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c9d34640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c40c79e0 con 0x7f04c40d19f0 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c9d34640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f04c40c7cc0 con 0x7f04c40cf080 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04c40cf080 0x7f04c4166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c37fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f04c41459c0 0x7f04c40c6e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36104/0 (socket says 172.21.15.111:36104) 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c37fe640 1 -- 172.21.15.111:0/527426995 learned_addr learned my addr 172.21.15.111:0/527426995 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c3fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04c40cf080 0x7f04c4166fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36500/0 (socket says 172.21.15.111:36500) 2024-05-27T00:40:15.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c3fff640 1 -- 172.21.15.111:0/527426995 >> v1:172.21.15.153:6789/0 conn(0x7f04c40d19f0 legacy=0x7f04c40ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:15.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c3fff640 1 -- 172.21.15.111:0/527426995 >> v1:172.21.15.111:6789/0 conn(0x7f04c41459c0 legacy=0x7f04c40c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.824+0000 7f04c3fff640 1 -- 172.21.15.111:0/527426995 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c40c7fa0 con 0x7f04c40cf080 2024-05-27T00:40:15.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.825+0000 7f04c3fff640 1 --2- 172.21.15.111:0/527426995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40cf080 0x7f04c4166fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0097a0 tx=0x7f04bc02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbe28455b948cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.825+0000 7f04c27fc640 1 -- 172.21.15.111:0/527426995 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc037070 con 0x7f04c40cf080 2024-05-27T00:40:15.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.825+0000 7f04c27fc640 1 -- 172.21.15.111:0/527426995 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc02acd0 con 0x7f04c40cf080 2024-05-27T00:40:15.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.825+0000 7f04c27fc640 1 -- 172.21.15.111:0/527426995 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc032660 con 0x7f04c40cf080 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 -- 172.21.15.111:0/527426995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40cf080 msgr2=0x7f04c4166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 --2- 172.21.15.111:0/527426995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40cf080 0x7f04c4166fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0097a0 tx=0x7f04bc02a040 comp rx=0 tx=0).stop 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 -- 172.21.15.111:0/527426995 shutdown_connections 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 --2- 172.21.15.111:0/527426995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40cf080 0x7f04c4166fe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 -- 172.21.15.111:0/527426995 >> 172.21.15.111:0/527426995 conn(0x7f04c40d1f60 msgr2=0x7f04c40d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 -- 172.21.15.111:0/527426995 shutdown_connections 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.826+0000 7f04c9d34640 1 -- 172.21.15.111:0/527426995 wait complete. 2024-05-27T00:40:15.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.827+0000 7f04c9d34640 1 Processor -- start 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.827+0000 7f04c9d34640 1 -- start start 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c9d34640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04c40cf080 0x7f04c40aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c9d34640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 0x7f04c406d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c9d34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41459c0 0x7f04c40b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c9d34640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40cdae0 con 0x7f04c40d19f0 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c9d34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40cd960 con 0x7f04c41459c0 2024-05-27T00:40:15.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c9d34640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40cdc60 con 0x7f04c40cf080 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 0x7f04c406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 0x7f04c406d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36506/0 (socket says 172.21.15.111:36506) 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 -- 172.21.15.111:0/2056647779 learned_addr learned my addr 172.21.15.111:0/2056647779 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 -- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04c40cf080 msgr2=0x7f04c40aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04c40cf080 0x7f04c40aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 -- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41459c0 msgr2=0x7f04c40b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41459c0 0x7f04c40b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 -- 172.21.15.111:0/2056647779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c406dca0 con 0x7f04c40d19f0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04c2ffd640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 0x7f04c406d170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c430 tx=0x7f04b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8011790 con 0x7f04c40d19f0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.828+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b8017070 con 0x7f04c40d19f0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.829+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8011b70 con 0x7f04c40d19f0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.829+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c406df80 con 0x7f04c40d19f0 2024-05-27T00:40:15.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.829+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c406e540 con 0x7f04c40d19f0 2024-05-27T00:40:15.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.829+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c406e740 con 0x7f04c40d19f0 2024-05-27T00:40:15.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.830+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04b8011d10 con 0x7f04c40d19f0 2024-05-27T00:40:15.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.830+0000 7f04abfff640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f04b006be30 0x7f04b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:15.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.831+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f04b808b0f0 con 0x7f04c40d19f0 2024-05-27T00:40:15.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.831+0000 7f04c37fe640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f04b006be30 0x7f04b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:15.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.831+0000 7f04c37fe640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f04b006be30 0x7f04b006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001db0 tx=0x7f04b4007d20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:15.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:15.835+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f04c406e740 con 0x7f04c40d19f0 2024-05-27T00:40:15.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:40:15.973 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:40:16.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.047+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04c40c8440 con 0x7f04c40d19f0 2024-05-27T00:40:16.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.047+0000 7f04abfff640 1 -- 172.21.15.111:0/2056647779 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f04b8059680 con 0x7f04c40d19f0 2024-05-27T00:40:16.048 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:16.049 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:40:16.049 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:40:16.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f04b006be30 msgr2=0x7f04b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f04b006be30 0x7f04b006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001db0 tx=0x7f04b4007d20 comp rx=0 tx=0).stop 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 msgr2=0x7f04c406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 0x7f04c406d170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c430 tx=0x7f04b800c8f0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 shutdown_connections 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f04b006be30 0x7f04b006e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41459c0 0x7f04c40b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04c40d19f0 0x7f04c406d170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 --2- 172.21.15.111:0/2056647779 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f04c40cf080 0x7f04c40aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 >> 172.21.15.111:0/2056647779 conn(0x7f04c40d1f60 msgr2=0x7f04c4067100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.049+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 shutdown_connections 2024-05-27T00:40:16.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.050+0000 7f04c9d34640 1 -- 172.21.15.111:0/2056647779 wait complete. 2024-05-27T00:40:16.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:16.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.200+0000 7f95679a9640 1 Processor -- start 2024-05-27T00:40:16.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.201+0000 7f95679a9640 1 -- start start 2024-05-27T00:40:16.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.201+0000 7f95679a9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 0x7f95601622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.201+0000 7f95679a9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956006cc70 con 0x7f95600b3630 2024-05-27T00:40:16.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.201+0000 7f95679a9640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95600d1e50 con 0x7f95600d2c10 2024-05-27T00:40:16.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.201+0000 7f95679a9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f95601627f0 con 0x7f9560161e50 2024-05-27T00:40:16.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f9565745640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f95600b3630 0x7f95600b72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36116/0 (socket says 172.21.15.111:36116) 2024-05-27T00:40:16.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f9565745640 1 -- 172.21.15.111:0/653242659 learned_addr learned my addr 172.21.15.111:0/653242659 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f9564f44640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 0x7f95601622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907812317 0 0) 0x7f956006cc70 con 0x7f95600b3630 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9548003520 con 0x7f95600b3630 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675631847 0 0) 0x7f95600d1e50 con 0x7f95600d2c10 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f956006cc70 con 0x7f95600d2c10 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1028818555 0 0) 0x7f9548003520 con 0x7f95600b3630 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 msgr2=0x7f95601622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 0x7f95601622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 >> v1:172.21.15.153:6789/0 conn(0x7f95600d2c10 legacy=0x7f95600d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95600d2130 con 0x7f95600b3630 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f9558001670 con 0x7f95600b3630 2024-05-27T00:40:16.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 >> v1:172.21.15.111:6789/0 conn(0x7f95600b3630 legacy=0x7f95600b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 0x7f9548003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95480040d0 0x7f9548004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9548004d40 0x7f95480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f954800ba70 con 0x7f95480040d0 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f954800bc20 con 0x7f9548004d40 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f954800be20 con 0x7f9560161e50 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f9564f44640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95480040d0 0x7f9548004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f9565745640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 0x7f9548003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.202+0000 7f9565f46640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9548004d40 0x7f95480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9564f44640 1 -- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 msgr2=0x7f9548003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9564f44640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9560161e50 0x7f9548003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9564f44640 1 -- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9548004d40 msgr2=0x7f95480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:16.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9564f44640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9548004d40 0x7f95480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9564f44640 1 -- 172.21.15.111:0/653242659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9548003520 con 0x7f95480040d0 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9565f46640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9548004d40 0x7f95480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f9564f44640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95480040d0 0x7f9548004800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9554009ac0 tx=0x7f9548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09141d3beaffa1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554036070 con 0x7f95480040d0 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9554031400 con 0x7f95480040d0 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.203+0000 7f955ffff640 1 -- 172.21.15.111:0/653242659 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554009690 con 0x7f95480040d0 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f95679a9640 1 -- 172.21.15.111:0/653242659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95480040d0 msgr2=0x7f9548004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f95679a9640 1 --2- 172.21.15.111:0/653242659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95480040d0 0x7f9548004800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9554009ac0 tx=0x7f9548005830 comp rx=0 tx=0).stop 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f9565f46640 1 -- 172.21.15.111:0/653242659 reap_dead start 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f95679a9640 1 -- 172.21.15.111:0/653242659 shutdown_connections 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f95679a9640 1 -- 172.21.15.111:0/653242659 >> 172.21.15.111:0/653242659 conn(0x7f95600d42c0 msgr2=0x7f95600d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f95679a9640 1 -- 172.21.15.111:0/653242659 shutdown_connections 2024-05-27T00:40:16.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.204+0000 7f95679a9640 1 -- 172.21.15.111:0/653242659 wait complete. 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.205+0000 7f95679a9640 1 Processor -- start 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.205+0000 7f95679a9640 1 -- start start 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f95679a9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95480040d0 0x7f956006bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f95679a9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95600cd990 0x7f95600cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f95679a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 0x7f956014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f95679a9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95600b79b0 con 0x7f956014ef90 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f95679a9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f95600d26f0 con 0x7f95600cd990 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f95679a9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f95601622b0 con 0x7f95480040d0 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 0x7f956014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 0x7f956014f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36520/0 (socket says 172.21.15.111:36520) 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 -- 172.21.15.111:0/578063952 learned_addr learned my addr 172.21.15.111:0/578063952 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565745640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95480040d0 0x7f956006bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9564f44640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95600cd990 0x7f95600cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 -- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95480040d0 msgr2=0x7f956006bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95480040d0 0x7f956006bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 -- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95600cd990 msgr2=0x7f95600cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95600cd990 0x7f95600cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 -- 172.21.15.111:0/578063952 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956015a2f0 con 0x7f956014ef90 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.206+0000 7f9565f46640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 0x7f956014f410 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9550001330 tx=0x7f955000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.207+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550019070 con 0x7f956014ef90 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.207+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955000ee50 con 0x7f956014ef90 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.207+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956015a5d0 con 0x7f956014ef90 2024-05-27T00:40:16.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.207+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550003dc0 con 0x7f956014ef90 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.207+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956015ab90 con 0x7f956014ef90 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.208+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9560068680 con 0x7f956014ef90 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.208+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f955001c070 con 0x7f956014ef90 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.209+0000 7f955dffb640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f954c06be30 0x7f954c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.209+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f955008a570 con 0x7f956014ef90 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.209+0000 7f9565745640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f954c06be30 0x7f954c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.209+0000 7f9565745640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f954c06be30 0x7f954c06e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9558007d80 tx=0x7f9558001be0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.213+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95500584b0 con 0x7f956014ef90 2024-05-27T00:40:16.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.387+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95600cef10 con 0x7f956014ef90 2024-05-27T00:40:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.388+0000 7f955dffb640 1 -- 172.21.15.111:0/578063952 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f955005c7b0 con 0x7f956014ef90 2024-05-27T00:40:16.389 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:16.389 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f954c06be30 msgr2=0x7f954c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f954c06be30 0x7f954c06e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9558007d80 tx=0x7f9558001be0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 msgr2=0x7f956014f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 0x7f956014f410 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9550001330 tx=0x7f955000ec00 comp rx=0 tx=0).stop 2024-05-27T00:40:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 shutdown_connections 2024-05-27T00:40:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f954c06be30 0x7f954c06e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f956014ef90 0x7f956014f410 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95600cd990 0x7f95600cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 --2- 172.21.15.111:0/578063952 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f95480040d0 0x7f956006bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 >> 172.21.15.111:0/578063952 conn(0x7f95600d42c0 msgr2=0x7f95600b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 shutdown_connections 2024-05-27T00:40:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.390+0000 7f95679a9640 1 -- 172.21.15.111:0/578063952 wait complete. 2024-05-27T00:40:16.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:39:18.903481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:39:19.886511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T00:39:23.607133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T00:40:16.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:16.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.504+0000 7f6e2d638640 1 Processor -- start 2024-05-27T00:40:16.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.504+0000 7f6e2d638640 1 -- start start 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e2d638640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2815c610 0x7f6e2815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e2d638640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e280b4ee0 0x7f6e28165ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e2d638640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e280ce470 con 0x7f6e2806fd90 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e2d638640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f6e280ce240 con 0x7f6e280b4ee0 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e2d638640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6e280b3a40 con 0x7f6e2815c610 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e280b4ee0 0x7f6e28165ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6e280b4ee0 0x7f6e28165ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36532/0 (socket says 172.21.15.111:36532) 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 -- 172.21.15.111:0/1767745456 learned_addr learned my addr 172.21.15.111:0/1767745456 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:16.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e26ffd640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2815c610 0x7f6e2815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 -- 172.21.15.111:0/1767745456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2815c610 msgr2=0x7f6e2815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2815c610 0x7f6e2815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 -- 172.21.15.111:0/1767745456 >> v1:172.21.15.153:6789/0 conn(0x7f6e2806fd90 legacy=0x7f6e2815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e277fe640 1 -- 172.21.15.111:0/1767745456 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e280ce750 con 0x7f6e280b4ee0 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.505+0000 7f6e26ffd640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2815c610 0x7f6e2815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.506+0000 7f6e277fe640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e280b4ee0 0x7f6e28165ba0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009e50 tx=0x7f6e2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c327478f83afb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.506+0000 7f6e25ffb640 1 -- 172.21.15.111:0/1767745456 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20036070 con 0x7f6e280b4ee0 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.506+0000 7f6e25ffb640 1 -- 172.21.15.111:0/1767745456 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e20031600 con 0x7f6e280b4ee0 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.506+0000 7f6e25ffb640 1 -- 172.21.15.111:0/1767745456 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200319d0 con 0x7f6e280b4ee0 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 -- 172.21.15.111:0/1767745456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e280b4ee0 msgr2=0x7f6e28165ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e280b4ee0 0x7f6e28165ba0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009e50 tx=0x7f6e2002a680 comp rx=0 tx=0).stop 2024-05-27T00:40:16.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 -- 172.21.15.111:0/1767745456 shutdown_connections 2024-05-27T00:40:16.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e280b4ee0 0x7f6e28165ba0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 --2- 172.21.15.111:0/1767745456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e2815c610 0x7f6e2815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 -- 172.21.15.111:0/1767745456 >> 172.21.15.111:0/1767745456 conn(0x7f6e28072a30 msgr2=0x7f6e2806bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 -- 172.21.15.111:0/1767745456 shutdown_connections 2024-05-27T00:40:16.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.507+0000 7f6e2d638640 1 -- 172.21.15.111:0/1767745456 wait complete. 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.508+0000 7f6e2d638640 1 Processor -- start 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.508+0000 7f6e2d638640 1 -- start start 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e2d638640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 0x7f6e28151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e2d638640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e280b4ee0 0x7f6e281542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e2d638640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2815c610 0x7f6e281547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e2d638640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2815c300 con 0x7f6e2806fd90 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e2d638640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f6e280ced10 con 0x7f6e2815c610 2024-05-27T00:40:16.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e2d638640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f6e281f1980 con 0x7f6e280b4ee0 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 0x7f6e28151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 0x7f6e28151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36538/0 (socket says 172.21.15.111:36538) 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 -- 172.21.15.111:0/2756580271 learned_addr learned my addr 172.21.15.111:0/2756580271 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e267fc640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e280b4ee0 0x7f6e281542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e277fe640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2815c610 0x7f6e281547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 -- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e280b4ee0 msgr2=0x7f6e281542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e280b4ee0 0x7f6e281542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 -- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2815c610 msgr2=0x7f6e281547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2815c610 0x7f6e281547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.509+0000 7f6e26ffd640 1 -- 172.21.15.111:0/2756580271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e281f5c60 con 0x7f6e2806fd90 2024-05-27T00:40:16.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.510+0000 7f6e26ffd640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 0x7f6e28151570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800e7e0 tx=0x7f6e1800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.510+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800f040 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.510+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18004030 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.510+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e281f1c40 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.510+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18004400 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.510+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e281f2200 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.511+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e28068680 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.511+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e1800c6d0 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.512+0000 7f6e0f7fe640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6e1406be30 0x7f6e1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.512+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f6e1808aff0 con 0x7f6e2806fd90 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.512+0000 7f6e267fc640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6e1406be30 0x7f6e1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.512+0000 7f6e267fc640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6e1406be30 0x7f6e1406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e28151fd0 tx=0x7f6e1c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.516+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6e18017020 con 0x7f6e2806fd90 2024-05-27T00:40:16.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.689+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e2815b8f0 con 0x7f6e2806fd90 2024-05-27T00:40:16.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.689+0000 7f6e0f7fe640 1 -- 172.21.15.111:0/2756580271 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f6e18058ef0 con 0x7f6e2806fd90 2024-05-27T00:40:16.691 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:16.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6e1406be30 msgr2=0x7f6e1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6e1406be30 0x7f6e1406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e28151fd0 tx=0x7f6e1c009040 comp rx=0 tx=0).stop 2024-05-27T00:40:16.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 msgr2=0x7f6e28151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 0x7f6e28151570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800e7e0 tx=0x7f6e1800eca0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 shutdown_connections 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f6e1406be30 0x7f6e1406e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e2815c610 0x7f6e281547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f6e280b4ee0 0x7f6e281542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 --2- 172.21.15.111:0/2756580271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e2806fd90 0x7f6e28151570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 >> 172.21.15.111:0/2756580271 conn(0x7f6e28072a30 msgr2=0x7f6e2815de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 shutdown_connections 2024-05-27T00:40:16.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.691+0000 7f6e2d638640 1 -- 172.21.15.111:0/2756580271 wait complete. 2024-05-27T00:40:16.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T00:40:16.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.804+0000 7fab82c9e640 1 Processor -- start 2024-05-27T00:40:16.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.804+0000 7fab82c9e640 1 -- start start 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.804+0000 7fab82c9e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fab7c06bc20 0x7fab7c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.804+0000 7fab82c9e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7c071d10 con 0x7fab7c0cf030 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.804+0000 7fab82c9e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7c071fe0 con 0x7fab7c151740 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.804+0000 7fab82c9e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fab7c0722c0 con 0x7fab7c06bc20 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab8123b640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fab7c0cf030 0x7fab7c0cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36126/0 (socket says 172.21.15.111:36126) 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab8123b640 1 -- 172.21.15.111:0/3338203648 learned_addr learned my addr 172.21.15.111:0/3338203648 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7bfff640 1 --2- 172.21.15.111:0/3338203648 >> v2:172.21.15.153:3300/0 conn(0x7fab7c06bc20 0x7fab7c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131888381 0 0) 0x7fab7c071d10 con 0x7fab7c0cf030 2024-05-27T00:40:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab64003520 con 0x7fab7c0cf030 2024-05-27T00:40:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666847806 0 0) 0x7fab7c071fe0 con 0x7fab7c151740 2024-05-27T00:40:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab7c071d10 con 0x7fab7c151740 2024-05-27T00:40:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7bfff640 1 -- 172.21.15.111:0/3338203648 >> v1:172.21.15.153:6789/0 conn(0x7fab7c151740 legacy=0x7fab7c071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7bfff640 1 -- 172.21.15.111:0/3338203648 >> v1:172.21.15.111:6789/0 conn(0x7fab7c0cf030 legacy=0x7fab7c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.805+0000 7fab7bfff640 1 -- 172.21.15.111:0/3338203648 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c0725a0 con 0x7fab7c06bc20 2024-05-27T00:40:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab7bfff640 1 --2- 172.21.15.111:0/3338203648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c06c0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab700097a0 tx=0x7fab7002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481e12c0bc4b3db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70037070 con 0x7fab7c06bc20 2024-05-27T00:40:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab70032440 con 0x7fab7c06bc20 2024-05-27T00:40:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab7b7fe640 1 -- 172.21.15.111:0/3338203648 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70032810 con 0x7fab7c06bc20 2024-05-27T00:40:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab82c9e640 1 -- 172.21.15.111:0/3338203648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 msgr2=0x7fab7c06c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3338203648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c06c0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab700097a0 tx=0x7fab7002a230 comp rx=0 tx=0).stop 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab82c9e640 1 -- 172.21.15.111:0/3338203648 shutdown_connections 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3338203648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c06c0a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.806+0000 7fab82c9e640 1 -- 172.21.15.111:0/3338203648 >> 172.21.15.111:0/3338203648 conn(0x7fab7c0b1c00 msgr2=0x7fab7c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.807+0000 7fab82c9e640 1 -- 172.21.15.111:0/3338203648 shutdown_connections 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.807+0000 7fab82c9e640 1 -- 172.21.15.111:0/3338203648 wait complete. 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.808+0000 7fab82c9e640 1 Processor -- start 2024-05-27T00:40:16.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.808+0000 7fab82c9e640 1 -- start start 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.808+0000 7fab82c9e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c1e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 0x7fab7c14d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab7c151740 0x7fab7c141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c151b20 con 0x7fab7c0cf030 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c0cfb90 con 0x7fab7c06bc20 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fab7c15af10 con 0x7fab7c151740 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 0x7fab7c14d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 0x7fab7c14d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36540/0 (socket says 172.21.15.111:36540) 2024-05-27T00:40:16.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 -- 172.21.15.111:0/3721287059 learned_addr learned my addr 172.21.15.111:0/3721287059 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab80a3a640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c1e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab8123b640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab7c151740 0x7fab7c141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 -- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab7c151740 msgr2=0x7fab7c141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab7c151740 0x7fab7c141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 -- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 msgr2=0x7fab7c1e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c1e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 -- 172.21.15.111:0/3721287059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c14a2c0 con 0x7fab7c0cf030 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab7bfff640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 0x7fab7c14d740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab700098c0 tx=0x7fab700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70041030 con 0x7fab7c0cf030 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7003d4e0 con 0x7fab7c0cf030 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70037040 con 0x7fab7c0cf030 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c1e4a70 con 0x7fab7c0cf030 2024-05-27T00:40:16.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.809+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1e5030 con 0x7fab7c0cf030 2024-05-27T00:40:16.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.810+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c068680 con 0x7fab7c0cf030 2024-05-27T00:40:16.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.811+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab70044070 con 0x7fab7c0cf030 2024-05-27T00:40:16.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.811+0000 7fab797fa640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fab6806be60 0x7fab6806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:16.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.811+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7fab700ab7d0 con 0x7fab7c0cf030 2024-05-27T00:40:16.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.811+0000 7fab80a3a640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fab6806be60 0x7fab6806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:16.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.812+0000 7fab80a3a640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fab6806be60 0x7fab6806e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab6c007fd0 tx=0x7fab6c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:16.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.815+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fab70079d70 con 0x7fab7c0cf030 2024-05-27T00:40:16.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.988+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab7c0af700 con 0x7fab7c0cf030 2024-05-27T00:40:16.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.988+0000 7fab797fa640 1 -- 172.21.15.111:0/3721287059 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fab70079670 con 0x7fab7c0cf030 2024-05-27T00:40:16.989 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:16.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fab6806be60 msgr2=0x7fab6806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fab6806be60 0x7fab6806e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab6c007fd0 tx=0x7fab6c009040 comp rx=0 tx=0).stop 2024-05-27T00:40:16.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 msgr2=0x7fab7c14d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:16.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 0x7fab7c14d740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab700098c0 tx=0x7fab700046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 shutdown_connections 2024-05-27T00:40:16.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fab6806be60 0x7fab6806e320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fab7c151740 0x7fab7c141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab7c0cf030 0x7fab7c14d740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 --2- 172.21.15.111:0/3721287059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab7c06bc20 0x7fab7c1e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:16.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 >> 172.21.15.111:0/3721287059 conn(0x7fab7c0b1c00 msgr2=0x7fab7c06b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:16.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 shutdown_connections 2024-05-27T00:40:16.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:16.990+0000 7fab82c9e640 1 -- 172.21.15.111:0/3721287059 wait complete. 2024-05-27T00:40:17.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T00:40:17.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.104+0000 7f29c4314640 1 Processor -- start 2024-05-27T00:40:17.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.104+0000 7f29c4314640 1 -- start start 2024-05-27T00:40:17.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c4314640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29bc147b60 0x7f29bc0a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c4314640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 0x7f29bc070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c4314640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29bc0a99e0 con 0x7f29bc070570 2024-05-27T00:40:17.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c4314640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f29bc0709e0 con 0x7f29bc147b60 2024-05-27T00:40:17.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c4314640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc15f990 con 0x7f29bc15f250 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c18af640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 0x7f29bc070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c20b0640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29bc147b60 0x7f29bc0a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c28b1640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f29bc070570 0x7f29bc161dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:40666/0 (socket says 172.21.15.111:40666) 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c18af640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 0x7f29bc070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:33040/0 (socket says 172.21.15.111:33040) 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c28b1640 1 -- 172.21.15.111:0/2969174609 learned_addr learned my addr 172.21.15.111:0/2969174609 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c18af640 1 -- 172.21.15.111:0/2969174609 >> v2:172.21.15.153:3300/0 conn(0x7f29bc147b60 msgr2=0x7f29bc0a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c18af640 1 --2- 172.21.15.111:0/2969174609 >> v2:172.21.15.153:3300/0 conn(0x7f29bc147b60 0x7f29bc0a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c18af640 1 -- 172.21.15.111:0/2969174609 >> v1:172.21.15.153:6789/0 conn(0x7f29bc070570 legacy=0x7f29bc161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c18af640 1 -- 172.21.15.111:0/2969174609 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc15fbe0 con 0x7f29bc15f250 2024-05-27T00:40:17.107 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.105+0000 7f29c20b0640 1 --2- 172.21.15.111:0/2969174609 >> v2:172.21.15.153:3300/0 conn(0x7f29bc147b60 0x7f29bc0a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.106+0000 7f29c18af640 1 --2- 172.21.15.111:0/2969174609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 0x7f29bc070030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0097a0 tx=0x7f29ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296ce11ab0aa4ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.106+0000 7f29c10ae640 1 -- 172.21.15.111:0/2969174609 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac037070 con 0x7f29bc15f250 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.106+0000 7f29c10ae640 1 -- 172.21.15.111:0/2969174609 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ac032440 con 0x7f29bc15f250 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.106+0000 7f29c10ae640 1 -- 172.21.15.111:0/2969174609 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac032810 con 0x7f29bc15f250 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 -- 172.21.15.111:0/2969174609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 msgr2=0x7f29bc070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 --2- 172.21.15.111:0/2969174609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 0x7f29bc070030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0097a0 tx=0x7f29ac02a180 comp rx=0 tx=0).stop 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 -- 172.21.15.111:0/2969174609 shutdown_connections 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 --2- 172.21.15.111:0/2969174609 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc15f250 0x7f29bc070030 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 --2- 172.21.15.111:0/2969174609 >> v2:172.21.15.153:3300/0 conn(0x7f29bc147b60 0x7f29bc0a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 -- 172.21.15.111:0/2969174609 >> 172.21.15.111:0/2969174609 conn(0x7f29bc073130 msgr2=0x7f29bc0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 -- 172.21.15.111:0/2969174609 shutdown_connections 2024-05-27T00:40:17.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.107+0000 7f29c4314640 1 -- 172.21.15.111:0/2969174609 wait complete. 2024-05-27T00:40:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.108+0000 7f29c4314640 1 Processor -- start 2024-05-27T00:40:17.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.108+0000 7f29c4314640 1 -- start start 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c4314640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc070570 0x7f29bc0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c4314640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc147b60 0x7f29bc1349e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c4314640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 0x7f29bc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c4314640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1616d0 con 0x7f29bc15f250 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c4314640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc15f6d0 con 0x7f29bc070570 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c4314640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc161850 con 0x7f29bc147b60 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 0x7f29bc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 0x7f29bc0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36556/0 (socket says 172.21.15.111:36556) 2024-05-27T00:40:17.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c20b0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc070570 0x7f29bc0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 -- 172.21.15.111:0/1664510845 learned_addr learned my addr 172.21.15.111:0/1664510845 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c18af640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc147b60 0x7f29bc1349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 -- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc147b60 msgr2=0x7f29bc1349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc147b60 0x7f29bc1349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 -- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc070570 msgr2=0x7f29bc0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc070570 0x7f29bc0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 -- 172.21.15.111:0/1664510845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc1354a0 con 0x7f29bc15f250 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c18af640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc147b60 0x7f29bc1349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:17.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c20b0640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc070570 0x7f29bc0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29c28b1640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 0x7f29bc0b3fd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29b400c480 tx=0x7f29b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.109+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4017070 con 0x7f29bc15f250 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.110+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b4005ce0 con 0x7f29bc15f250 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.110+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc135780 con 0x7f29bc15f250 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.110+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4007270 con 0x7f29bc15f250 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.110+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc135d40 con 0x7f29bc15f250 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.111+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc068680 con 0x7f29bc15f250 2024-05-27T00:40:17.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.111+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29b401a070 con 0x7f29bc15f250 2024-05-27T00:40:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.112+0000 7f29b2ffd640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f29a006be60 0x7f29a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.112+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f29b408a400 con 0x7f29bc15f250 2024-05-27T00:40:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.112+0000 7f29c20b0640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f29a006be60 0x7f29a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.112+0000 7f29c20b0640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f29a006be60 0x7f29a006e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009fd0 tx=0x7f29b800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.116+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f29b40583d0 con 0x7f29bc15f250 2024-05-27T00:40:17.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.288+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f29bc146b10 con 0x7f29bc15f250 2024-05-27T00:40:17.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.289+0000 7f29b2ffd640 1 -- 172.21.15.111:0/1664510845 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f29b405c6d0 con 0x7f29bc15f250 2024-05-27T00:40:17.290 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f29a006be60 msgr2=0x7f29a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f29a006be60 0x7f29a006e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009fd0 tx=0x7f29b800b040 comp rx=0 tx=0).stop 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 msgr2=0x7f29bc0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 0x7f29bc0b3fd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29b400c480 tx=0x7f29b400c940 comp rx=0 tx=0).stop 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 shutdown_connections 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f29a006be60 0x7f29a006e320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc15f250 0x7f29bc0b3fd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f29bc147b60 0x7f29bc1349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 --2- 172.21.15.111:0/1664510845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc070570 0x7f29bc0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 >> 172.21.15.111:0/1664510845 conn(0x7f29bc073130 msgr2=0x7f29bc146df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:17.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 shutdown_connections 2024-05-27T00:40:17.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.291+0000 7f29c4314640 1 -- 172.21.15.111:0/1664510845 wait complete. 2024-05-27T00:40:17.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T00:40:17.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.405+0000 7f22124c1640 1 Processor -- start 2024-05-27T00:40:17.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.405+0000 7f22124c1640 1 -- start start 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f22124c1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f220c06b390 0x7f220c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f22124c1640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f220c0cdc80 0x7f220c0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f22124c1640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220c0b36e0 con 0x7f220c144000 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f22124c1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f220c0b42f0 con 0x7f220c06b390 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f22124c1640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f220c0b39c0 con 0x7f220c0cdc80 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f2203fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f220c06b390 0x7f220c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f2203fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f220c06b390 0x7f220c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36564/0 (socket says 172.21.15.111:36564) 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.406+0000 7f2203fff640 1 -- 172.21.15.111:0/2892497032 learned_addr learned my addr 172.21.15.111:0/2892497032 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.407+0000 7f2210a5e640 1 --2- 172.21.15.111:0/2892497032 >> v2:172.21.15.153:3300/0 conn(0x7f220c0cdc80 0x7f220c0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.407+0000 7f2203fff640 1 -- 172.21.15.111:0/2892497032 >> v2:172.21.15.153:3300/0 conn(0x7f220c0cdc80 msgr2=0x7f220c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.407+0000 7f2203fff640 1 --2- 172.21.15.111:0/2892497032 >> v2:172.21.15.153:3300/0 conn(0x7f220c0cdc80 0x7f220c0726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.407+0000 7f2203fff640 1 -- 172.21.15.111:0/2892497032 >> v1:172.21.15.153:6789/0 conn(0x7f220c144000 legacy=0x7f220c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.407+0000 7f2203fff640 1 -- 172.21.15.111:0/2892497032 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c0b3ca0 con 0x7f220c06b390 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.407+0000 7f2203fff640 1 --2- 172.21.15.111:0/2892497032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c06b390 0x7f220c0cd740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f220c06e340 tx=0x7f21f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c193acc4861bd55a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f220b7fe640 1 -- 172.21.15.111:0/2892497032 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037070 con 0x7f220c06b390 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f220b7fe640 1 -- 172.21.15.111:0/2892497032 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4032440 con 0x7f220c06b390 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f220b7fe640 1 -- 172.21.15.111:0/2892497032 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4032810 con 0x7f220c06b390 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 -- 172.21.15.111:0/2892497032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c06b390 msgr2=0x7f220c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 --2- 172.21.15.111:0/2892497032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c06b390 0x7f220c0cd740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f220c06e340 tx=0x7f21f402a270 comp rx=0 tx=0).stop 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 -- 172.21.15.111:0/2892497032 shutdown_connections 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 --2- 172.21.15.111:0/2892497032 >> v2:172.21.15.153:3300/0 conn(0x7f220c0cdc80 0x7f220c0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 --2- 172.21.15.111:0/2892497032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c06b390 0x7f220c0cd740 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 -- 172.21.15.111:0/2892497032 >> 172.21.15.111:0/2892497032 conn(0x7f220c0b1c00 msgr2=0x7f220c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:17.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 -- 172.21.15.111:0/2892497032 shutdown_connections 2024-05-27T00:40:17.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.408+0000 7f22124c1640 1 -- 172.21.15.111:0/2892497032 wait complete. 2024-05-27T00:40:17.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.409+0000 7f22124c1640 1 Processor -- start 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.409+0000 7f22124c1640 1 -- start start 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f22124c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220c06b390 0x7f220c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f22124c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 0x7f220c140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f22124c1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f220c144000 0x7f220c13cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f22124c1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f220c072d90 con 0x7f220c0cdc80 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f22124c1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f220c072c10 con 0x7f220c06b390 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f22124c1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f220c072f10 con 0x7f220c144000 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f2203fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 0x7f220c140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f2203fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 0x7f220c140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36572/0 (socket says 172.21.15.111:36572) 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f2203fff640 1 -- 172.21.15.111:0/1075629019 learned_addr learned my addr 172.21.15.111:0/1075629019 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f220bfff640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220c06b390 0x7f220c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.410+0000 7f2210a5e640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f220c144000 0x7f220c13cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2203fff640 1 -- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f220c144000 msgr2=0x7f220c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2203fff640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f220c144000 0x7f220c13cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2203fff640 1 -- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220c06b390 msgr2=0x7f220c1f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2203fff640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220c06b390 0x7f220c1f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2203fff640 1 -- 172.21.15.111:0/1075629019 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c1f3e80 con 0x7f220c0cdc80 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2210a5e640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f220c144000 0x7f220c13cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f2203fff640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 0x7f220c140b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f21f402a2b0 tx=0x7f21f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4041020 con 0x7f220c0cdc80 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f40306c0 con 0x7f220c0cdc80 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037040 con 0x7f220c0cdc80 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c1f4160 con 0x7f220c0cdc80 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.411+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1f4720 con 0x7f220c0cdc80 2024-05-27T00:40:17.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.412+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c068680 con 0x7f220c0cdc80 2024-05-27T00:40:17.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.412+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21f4044020 con 0x7f220c0cdc80 2024-05-27T00:40:17.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.413+0000 7f22097fa640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f21ec06be60 0x7f21ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.413+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f21f40ab980 con 0x7f220c0cdc80 2024-05-27T00:40:17.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.413+0000 7f220bfff640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f21ec06be60 0x7f21ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.414+0000 7f220bfff640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f21ec06be60 0x7f21ec06e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f220c1380b0 tx=0x7f21fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.417+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f21f404b030 con 0x7f220c0cdc80 2024-05-27T00:40:17.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.593+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f220c06bd30 con 0x7f220c0cdc80 2024-05-27T00:40:17.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.593+0000 7f22097fa640 1 -- 172.21.15.111:0/1075629019 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f21f4079fa0 con 0x7f220c0cdc80 2024-05-27T00:40:17.594 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T00:40:17.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.595+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f21ec06be60 msgr2=0x7f21ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.595+0000 7f22124c1640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f21ec06be60 0x7f21ec06e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f220c1380b0 tx=0x7f21fc009040 comp rx=0 tx=0).stop 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.595+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 msgr2=0x7f220c140b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.595+0000 7f22124c1640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 0x7f220c140b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f21f402a2b0 tx=0x7f21f40046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 shutdown_connections 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f21ec06be60 0x7f21ec06e320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f220c144000 0x7f220c13cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f220c0cdc80 0x7f220c140b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 --2- 172.21.15.111:0/1075629019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f220c06b390 0x7f220c1f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 >> 172.21.15.111:0/1075629019 conn(0x7f220c0b1c00 msgr2=0x7f220c06c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 shutdown_connections 2024-05-27T00:40:17.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.596+0000 7f22124c1640 1 -- 172.21.15.111:0/1075629019 wait complete. 2024-05-27T00:40:17.609 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:40:17.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.708+0000 7f14f7c53640 1 Processor -- start 2024-05-27T00:40:17.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.708+0000 7f14f7c53640 1 -- start start 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f7c53640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f14f0151740 0x7f14f0071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f7c53640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f0071b40 0x7f14f006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f7c53640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f0071fe0 con 0x7f14f006c160 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f7c53640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f14f006c5d0 con 0x7f14f0151740 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f7c53640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00722c0 con 0x7f14f0071b40 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f51ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f0071b40 0x7f14f006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f59ef640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f14f0151740 0x7f14f0071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f51ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f0071b40 0x7f14f006bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:33058/0 (socket says 172.21.15.111:33058) 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f51ee640 1 -- 172.21.15.111:0/2171807285 learned_addr learned my addr 172.21.15.111:0/2171807285 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.709+0000 7f14f61f0640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f14f006c160 0x7f14f00cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:40680/0 (socket says 172.21.15.111:40680) 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.710+0000 7f14f59ef640 1 -- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f0071b40 msgr2=0x7f14f006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.710+0000 7f14f59ef640 1 --2- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f0071b40 0x7f14f006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.710+0000 7f14f59ef640 1 -- 172.21.15.111:0/2171807285 >> v1:172.21.15.153:6789/0 conn(0x7f14f006c160 legacy=0x7f14f00cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.712 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.710+0000 7f14f59ef640 1 -- 172.21.15.111:0/2171807285 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f00725a0 con 0x7f14f0151740 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.710+0000 7f14f59ef640 1 --2- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f0071600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009d90 tx=0x7f14e002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=786b72ba4ac65d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f49ed640 1 -- 172.21.15.111:0/2171807285 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0007740 con 0x7f14f0151740 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f49ed640 1 -- 172.21.15.111:0/2171807285 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e00078e0 con 0x7f14f0151740 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f49ed640 1 -- 172.21.15.111:0/2171807285 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0007cb0 con 0x7f14f0151740 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 -- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 msgr2=0x7f14f0071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 --2- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f0071600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009d90 tx=0x7f14e002a660 comp rx=0 tx=0).stop 2024-05-27T00:40:17.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 -- 172.21.15.111:0/2171807285 shutdown_connections 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 --2- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f0071b40 0x7f14f006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 --2- 172.21.15.111:0/2171807285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f0071600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 -- 172.21.15.111:0/2171807285 >> 172.21.15.111:0/2171807285 conn(0x7f14f00b1c00 msgr2=0x7f14f00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 -- 172.21.15.111:0/2171807285 shutdown_connections 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.711+0000 7f14f7c53640 1 -- 172.21.15.111:0/2171807285 wait complete. 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.712+0000 7f14f7c53640 1 Processor -- start 2024-05-27T00:40:17.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.712+0000 7f14f7c53640 1 -- start start 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f7c53640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f006c160 0x7f14f01f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f7c53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 0x7f14f01f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f7c53640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f01f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f7c53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14f015af10 con 0x7f14f0071b40 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f7c53640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f14f0151b20 con 0x7f14f0151740 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f7c53640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f14f015b090 con 0x7f14f006c160 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f51ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 0x7f14f01f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f51ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 0x7f14f01f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36586/0 (socket says 172.21.15.111:36586) 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f51ee640 1 -- 172.21.15.111:0/131274372 learned_addr learned my addr 172.21.15.111:0/131274372 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f59ef640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f006c160 0x7f14f01f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.713+0000 7f14f61f0640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f01f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f51ee640 1 -- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f006c160 msgr2=0x7f14f01f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f51ee640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f006c160 0x7f14f01f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f51ee640 1 -- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 msgr2=0x7f14f01f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f51ee640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f01f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f51ee640 1 -- 172.21.15.111:0/131274372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f013c300 con 0x7f14f0071b40 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f51ee640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 0x7f14f01f4c00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f14e400e8e0 tx=0x7f14e400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4016070 con 0x7f14f0071b40 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e400cc40 con 0x7f14f0071b40 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e400b040 con 0x7f14f0071b40 2024-05-27T00:40:17.718 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f013c570 con 0x7f14f0071b40 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.714+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f013cb30 con 0x7f14f0071b40 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.715+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f0068680 con 0x7f14f0071b40 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.715+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14e4019070 con 0x7f14f0071b40 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.716+0000 7f14ee7fc640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f14dc06be30 0x7f14dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.716+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f14e408ad50 con 0x7f14f0071b40 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.716+0000 7f14f59ef640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f14dc06be30 0x7f14dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:17.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.717+0000 7f14f59ef640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f14dc06be30 0x7f14dc06e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f14e0034000 tx=0x7f14e002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:17.721 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.720+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f14e408e030 con 0x7f14f0071b40 2024-05-27T00:40:17.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.929+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14f0150820 con 0x7f14f0071b40 2024-05-27T00:40:17.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.929+0000 7f14ee7fc640 1 -- 172.21.15.111:0/131274372 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f14e405f020 con 0x7f14f0071b40 2024-05-27T00:40:17.930 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:17.930 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:40:17.931 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:40:17.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f14dc06be30 msgr2=0x7f14dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f14dc06be30 0x7f14dc06e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f14e0034000 tx=0x7f14e002f000 comp rx=0 tx=0).stop 2024-05-27T00:40:17.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 msgr2=0x7f14f01f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:17.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 0x7f14f01f4c00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f14e400e8e0 tx=0x7f14e400ece0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 shutdown_connections 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f14dc06be30 0x7f14dc06e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f14f0151740 0x7f14f01f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14f0071b40 0x7f14f01f4c00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 --2- 172.21.15.111:0/131274372 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f14f006c160 0x7f14f01f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 >> 172.21.15.111:0/131274372 conn(0x7f14f00b1c00 msgr2=0x7f14f0158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 shutdown_connections 2024-05-27T00:40:17.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:17.931+0000 7f14f7c53640 1 -- 172.21.15.111:0/131274372 wait complete. 2024-05-27T00:40:17.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:18.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.042+0000 7f9f6b56a640 1 Processor -- start 2024-05-27T00:40:18.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.042+0000 7f9f6b56a640 1 -- start start 2024-05-27T00:40:18.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f6b56a640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9f64147120 0x7f9f640b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f6b56a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f640cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f6b56a640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640c72b0 con 0x7f9f640c6e20 2024-05-27T00:40:18.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f6b56a640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9f640c7590 con 0x7f9f64147120 2024-05-27T00:40:18.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f6b56a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9f640c7870 con 0x7f9f640cf080 2024-05-27T00:40:18.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f68b05640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9f640c6e20 0x7f9f640ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36140/0 (socket says 172.21.15.111:36140) 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f68b05640 1 -- 172.21.15.111:0/4137138472 learned_addr learned my addr 172.21.15.111:0/4137138472 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69b07640 1 --2- 172.21.15.111:0/4137138472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f640cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69306640 1 --2- 172.21.15.111:0/4137138472 >> v2:172.21.15.153:3300/0 conn(0x7f9f64147120 0x7f9f640b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f5bfff640 1 -- 172.21.15.111:0/4137138472 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116188750 0 0) 0x7f9f640c72b0 con 0x7f9f640c6e20 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f5bfff640 1 -- 172.21.15.111:0/4137138472 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f50003520 con 0x7f9f640c6e20 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69b07640 1 -- 172.21.15.111:0/4137138472 >> v2:172.21.15.153:3300/0 conn(0x7f9f64147120 msgr2=0x7f9f640b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69b07640 1 --2- 172.21.15.111:0/4137138472 >> v2:172.21.15.153:3300/0 conn(0x7f9f64147120 0x7f9f640b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69b07640 1 -- 172.21.15.111:0/4137138472 >> v1:172.21.15.111:6789/0 conn(0x7f9f640c6e20 legacy=0x7f9f640ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69b07640 1 -- 172.21.15.111:0/4137138472 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640c7b50 con 0x7f9f640cf080 2024-05-27T00:40:18.045 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.043+0000 7f9f69306640 1 --2- 172.21.15.111:0/4137138472 >> v2:172.21.15.153:3300/0 conn(0x7f9f64147120 0x7f9f640b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.044+0000 7f9f69b07640 1 --2- 172.21.15.111:0/4137138472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f640cc5b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0097a0 tx=0x7f9f5c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65456367c822b2e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.044+0000 7f9f5bfff640 1 -- 172.21.15.111:0/4137138472 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c037070 con 0x7f9f640cf080 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.044+0000 7f9f5bfff640 1 -- 172.21.15.111:0/4137138472 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c032440 con 0x7f9f640cf080 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.044+0000 7f9f5bfff640 1 -- 172.21.15.111:0/4137138472 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0327f0 con 0x7f9f640cf080 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 -- 172.21.15.111:0/4137138472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 msgr2=0x7f9f640cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/4137138472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f640cc5b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0097a0 tx=0x7f9f5c02a180 comp rx=0 tx=0).stop 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 -- 172.21.15.111:0/4137138472 shutdown_connections 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/4137138472 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f640cc5b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/4137138472 >> v2:172.21.15.153:3300/0 conn(0x7f9f64147120 0x7f9f640b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 -- 172.21.15.111:0/4137138472 >> 172.21.15.111:0/4137138472 conn(0x7f9f640d1f60 msgr2=0x7f9f640d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 -- 172.21.15.111:0/4137138472 shutdown_connections 2024-05-27T00:40:18.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.045+0000 7f9f6b56a640 1 -- 172.21.15.111:0/4137138472 wait complete. 2024-05-27T00:40:18.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.046+0000 7f9f6b56a640 1 Processor -- start 2024-05-27T00:40:18.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.046+0000 7f9f6b56a640 1 -- start start 2024-05-27T00:40:18.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.046+0000 7f9f6b56a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f64157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f6b56a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 0x7f9f640b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f6b56a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f640b5490 0x7f9f641452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f6b56a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f641477d0 con 0x7f9f64147120 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f6b56a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640c8350 con 0x7f9f640b5490 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f6b56a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9f64147950 con 0x7f9f640cf080 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 0x7f9f640b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 0x7f9f640b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36598/0 (socket says 172.21.15.111:36598) 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 -- 172.21.15.111:0/2316289099 learned_addr learned my addr 172.21.15.111:0/2316289099 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f69306640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f64157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f69b07640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f640b5490 0x7f9f641452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 -- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 msgr2=0x7f9f64157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f64157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 -- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f640b5490 msgr2=0x7f9f641452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f640b5490 0x7f9f641452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 -- 172.21.15.111:0/2316289099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f64146030 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f68b05640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 0x7f9f640b4f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c390 tx=0x7f9f5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54017070 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f54005ce0 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6415f2f0 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.047+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f540072f0 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.048+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6415f8b0 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.048+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f640b58a0 con 0x7f9f64147120 2024-05-27T00:40:18.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.049+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f5401a070 con 0x7f9f64147120 2024-05-27T00:40:18.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.049+0000 7f9f59ffb640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9f4806be30 0x7f9f4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.049+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f9f5408a610 con 0x7f9f64147120 2024-05-27T00:40:18.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.050+0000 7f9f69306640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9f4806be30 0x7f9f4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.050+0000 7f9f69306640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9f4806be30 0x7f9f4806e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009bd0 tx=0x7f9f6000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.054+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9f640b58a0 con 0x7f9f64147120 2024-05-27T00:40:18.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.230+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f6406d0d0 con 0x7f9f64147120 2024-05-27T00:40:18.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.231+0000 7f9f59ffb640 1 -- 172.21.15.111:0/2316289099 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f9f54058550 con 0x7f9f64147120 2024-05-27T00:40:18.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:18.232 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9f4806be30 msgr2=0x7f9f4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9f4806be30 0x7f9f4806e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009bd0 tx=0x7f9f6000b040 comp rx=0 tx=0).stop 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 msgr2=0x7f9f640b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 0x7f9f640b4f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c390 tx=0x7f9f5400c850 comp rx=0 tx=0).stop 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 shutdown_connections 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9f4806be30 0x7f9f4806e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.233+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f640b5490 0x7f9f641452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.234+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f64147120 0x7f9f640b4f50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.234+0000 7f9f6b56a640 1 --2- 172.21.15.111:0/2316289099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9f640cf080 0x7f9f64157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.234+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 >> 172.21.15.111:0/2316289099 conn(0x7f9f640d1f60 msgr2=0x7f9f6406fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.234+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 shutdown_connections 2024-05-27T00:40:18.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.234+0000 7f9f6b56a640 1 -- 172.21.15.111:0/2316289099 wait complete. 2024-05-27T00:40:18.246 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-27T00:40:18.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:40:18.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.345+0000 7f5d3dc72640 1 Processor -- start 2024-05-27T00:40:18.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.345+0000 7f5d3dc72640 1 -- start start 2024-05-27T00:40:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d3dc72640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 0x7f5d380cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d3dc72640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d380cdc80 0x7f5d380726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d3dc72640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d380b36e0 con 0x7f5d38144000 2024-05-27T00:40:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d3dc72640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5d380b42f0 con 0x7f5d380cdc80 2024-05-27T00:40:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d3dc72640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5d380b39c0 con 0x7f5d3806b390 2024-05-27T00:40:18.349 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d377fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5d38144000 0x7f5d380b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36146/0 (socket says 172.21.15.111:36146) 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d377fe640 1 -- 172.21.15.111:0/182818088 learned_addr learned my addr 172.21.15.111:0/182818088 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d36ffd640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 0x7f5d380cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566837310 0 0) 0x7f5d380b36e0 con 0x7f5d38144000 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d37fff640 1 --2- 172.21.15.111:0/182818088 >> v2:172.21.15.153:3300/0 conn(0x7f5d380cdc80 0x7f5d380726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d20003520 con 0x7f5d38144000 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2439039465 0 0) 0x7f5d20003520 con 0x7f5d38144000 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 msgr2=0x7f5d380cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 0x7f5d380cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 >> v2:172.21.15.153:3300/0 conn(0x7f5d380cdc80 msgr2=0x7f5d380726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 --2- 172.21.15.111:0/182818088 >> v2:172.21.15.153:3300/0 conn(0x7f5d380cdc80 0x7f5d380726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d380b3ca0 con 0x7f5d38144000 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.346+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f5d28001670 con 0x7f5d38144000 2024-05-27T00:40:18.350 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 >> v1:172.21.15.111:6789/0 conn(0x7f5d38144000 legacy=0x7f5d380b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 0x7f5d20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d20004d40 0x7f5d200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380b42f0 con 0x7f5d200039b0 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380b39c0 con 0x7f5d20004d40 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2000b9c0 con 0x7f5d3806b390 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d377fe640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d36ffd640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 0x7f5d20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d37fff640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d20004d40 0x7f5d200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d377fe640 1 -- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 msgr2=0x7f5d20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d377fe640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d3806b390 0x7f5d20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.351 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d377fe640 1 -- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d20004d40 msgr2=0x7f5d200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d377fe640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d20004d40 0x7f5d200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d377fe640 1 -- 172.21.15.111:0/182818088 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d20003520 con 0x7f5d200039b0 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.347+0000 7f5d37fff640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d20004d40 0x7f5d200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d377fe640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800c2c0 tx=0x7f5d20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fac2c7542855c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28007d50 con 0x7f5d200039b0 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2802e070 con 0x7f5d200039b0 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d367fc640 1 -- 172.21.15.111:0/182818088 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28005d30 con 0x7f5d200039b0 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d3dc72640 1 -- 172.21.15.111:0/182818088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d200039b0 msgr2=0x7f5d20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/182818088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d200039b0 0x7f5d20003dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800c2c0 tx=0x7f5d20005830 comp rx=0 tx=0).stop 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d37fff640 1 -- 172.21.15.111:0/182818088 reap_dead start 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d3dc72640 1 -- 172.21.15.111:0/182818088 shutdown_connections 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d3dc72640 1 -- 172.21.15.111:0/182818088 >> 172.21.15.111:0/182818088 conn(0x7f5d380b1c00 msgr2=0x7f5d380b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:18.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d3dc72640 1 -- 172.21.15.111:0/182818088 shutdown_connections 2024-05-27T00:40:18.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.348+0000 7f5d3dc72640 1 -- 172.21.15.111:0/182818088 wait complete. 2024-05-27T00:40:18.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 Processor -- start 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 -- start start 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d200039b0 0x7f5d381546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 0x7f5d380ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d38152bf0 0x7f5d380c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380cd8c0 con 0x7f5d38152770 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380cd740 con 0x7f5d200039b0 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.350+0000 7f5d3dc72640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5d380cda40 con 0x7f5d38152bf0 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 0x7f5d380ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 0x7f5d380ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36616/0 (socket says 172.21.15.111:36616) 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 -- 172.21.15.111:0/3812018186 learned_addr learned my addr 172.21.15.111:0/3812018186 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d377fe640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d200039b0 0x7f5d381546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d37fff640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d38152bf0 0x7f5d380c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 -- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d38152bf0 msgr2=0x7f5d380c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d38152bf0 0x7f5d380c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 -- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d200039b0 msgr2=0x7f5d381546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d200039b0 0x7f5d381546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 -- 172.21.15.111:0/3812018186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d380ab540 con 0x7f5d38152770 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d37fff640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d38152bf0 0x7f5d380c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d377fe640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d200039b0 0x7f5d381546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d36ffd640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 0x7f5d380ae610 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c011910 tx=0x7f5d2c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00faf0 con 0x7f5d38152770 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2c00fc90 con 0x7f5d38152770 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c014440 con 0x7f5d38152770 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d380ab820 con 0x7f5d38152770 2024-05-27T00:40:18.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.351+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d380abde0 con 0x7f5d38152770 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.352+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d380b4510 con 0x7f5d38152770 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.353+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d2c012040 con 0x7f5d38152770 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.353+0000 7f5d1ffff640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5d2406be30 0x7f5d2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.353+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f5d2c08b960 con 0x7f5d38152770 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.353+0000 7f5d377fe640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5d2406be30 0x7f5d2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.354+0000 7f5d377fe640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5d2406be30 0x7f5d2406e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d28033000 tx=0x7f5d28009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.357+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5d380b4510 con 0x7f5d38152770 2024-05-27T00:40:18.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.567+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d38068680 con 0x7f5d38152770 2024-05-27T00:40:18.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.567+0000 7f5d1ffff640 1 -- 172.21.15.111:0/3812018186 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f5d2c019030 con 0x7f5d38152770 2024-05-27T00:40:18.568 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:18.568 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:40:18.569 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:40:18.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5d2406be30 msgr2=0x7f5d2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5d2406be30 0x7f5d2406e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d28033000 tx=0x7f5d28009030 comp rx=0 tx=0).stop 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 msgr2=0x7f5d380ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 0x7f5d380ae610 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c011910 tx=0x7f5d2c011dd0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 shutdown_connections 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5d2406be30 0x7f5d2406e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5d38152bf0 0x7f5d380c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38152770 0x7f5d380ae610 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 --2- 172.21.15.111:0/3812018186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d200039b0 0x7f5d381546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 >> 172.21.15.111:0/3812018186 conn(0x7f5d380b1c00 msgr2=0x7f5d380b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 shutdown_connections 2024-05-27T00:40:18.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.569+0000 7f5d3dc72640 1 -- 172.21.15.111:0/3812018186 wait complete. 2024-05-27T00:40:18.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:40:18.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 Processor -- start 2024-05-27T00:40:18.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 -- start start 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b3630 0x7ff7f00b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7f00b77e0 0x7ff7f0161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f006cc70 con 0x7ff7f0162390 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f00d1e50 con 0x7ff7f00b77e0 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.681+0000 7ff7f70bd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f00d2130 con 0x7ff7f00b3630 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7f565a640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff7f0162390 0x7ff7f00d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36154/0 (socket says 172.21.15.111:36154) 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7effff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7f00b77e0 0x7ff7f0161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7f565a640 1 -- 172.21.15.111:0/329938475 learned_addr learned my addr 172.21.15.111:0/329938475 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7f4e59640 1 --2- 172.21.15.111:0/329938475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b3630 0x7ff7f00b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7effff640 1 -- 172.21.15.111:0/329938475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b3630 msgr2=0x7ff7f00b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7effff640 1 --2- 172.21.15.111:0/329938475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b3630 0x7ff7f00b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7effff640 1 -- 172.21.15.111:0/329938475 >> v1:172.21.15.111:6789/0 conn(0x7ff7f0162390 legacy=0x7ff7f00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7effff640 1 -- 172.21.15.111:0/329938475 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f00d2410 con 0x7ff7f00b77e0 2024-05-27T00:40:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.682+0000 7ff7ef7fe640 1 -- 172.21.15.111:0/329938475 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687046585 0 0) 0x7ff7f006cc70 con 0x7ff7f0162390 2024-05-27T00:40:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7effff640 1 --2- 172.21.15.111:0/329938475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b77e0 0x7ff7f0161e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009d90 tx=0x7ff7e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb48c458dfacd99 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7ef7fe640 1 -- 172.21.15.111:0/329938475 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0007740 con 0x7ff7f00b77e0 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7ef7fe640 1 -- 172.21.15.111:0/329938475 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e00078e0 con 0x7ff7f00b77e0 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7ef7fe640 1 -- 172.21.15.111:0/329938475 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0007cb0 con 0x7ff7f00b77e0 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 -- 172.21.15.111:0/329938475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b77e0 msgr2=0x7ff7f0161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/329938475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b77e0 0x7ff7f0161e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009d90 tx=0x7ff7e002a660 comp rx=0 tx=0).stop 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 -- 172.21.15.111:0/329938475 shutdown_connections 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/329938475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b77e0 0x7ff7f0161e50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/329938475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b3630 0x7ff7f00b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 -- 172.21.15.111:0/329938475 >> 172.21.15.111:0/329938475 conn(0x7ff7f00d42c0 msgr2=0x7ff7f00d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 -- 172.21.15.111:0/329938475 shutdown_connections 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.683+0000 7ff7f70bd640 1 -- 172.21.15.111:0/329938475 wait complete. 2024-05-27T00:40:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.684+0000 7ff7f70bd640 1 Processor -- start 2024-05-27T00:40:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 -- start start 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 0x7ff7f01515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b77e0 0x7ff7f00aac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f0162390 0x7ff7f00ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00d34a0 con 0x7ff7f00b3630 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00d3320 con 0x7ff7f0162390 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f70bd640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f00d3620 con 0x7ff7f00b77e0 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f4e59640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 0x7ff7f01515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f4e59640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 0x7ff7f01515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36640/0 (socket says 172.21.15.111:36640) 2024-05-27T00:40:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.685+0000 7ff7f4e59640 1 -- 172.21.15.111:0/2471452379 learned_addr learned my addr 172.21.15.111:0/2471452379 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7effff640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b77e0 0x7ff7f00aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f565a640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f0162390 0x7ff7f00ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f4e59640 1 -- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b77e0 msgr2=0x7ff7f00aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f4e59640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b77e0 0x7ff7f00aac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f4e59640 1 -- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f0162390 msgr2=0x7ff7f00ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f4e59640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f0162390 0x7ff7f00ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f4e59640 1 -- 172.21.15.111:0/2471452379 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f00c8b00 con 0x7ff7f00b3630 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f4e59640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 0x7ff7f01515e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800d800 tx=0x7ff7e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800b660 con 0x7ff7f00b3630 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e8019070 con 0x7ff7f00b3630 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f00c8de0 con 0x7ff7f00b3630 2024-05-27T00:40:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800ba20 con 0x7ff7f00b3630 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.686+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f00c93a0 con 0x7ff7f00b3630 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.687+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f0068680 con 0x7ff7f00b3630 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.687+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7e800bbc0 con 0x7ff7f00b3630 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.688+0000 7ff7ed7fa640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff7d006be60 0x7ff7d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.688+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7ff7e808b610 con 0x7ff7f00b3630 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.688+0000 7ff7effff640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff7d006be60 0x7ff7d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.689+0000 7ff7effff640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff7d006be60 0x7ff7d006e320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0034000 tx=0x7ff7e002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:18.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.692+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff7e8059c30 con 0x7ff7f00b3630 2024-05-27T00:40:18.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.904+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f00cef10 con 0x7ff7f00b3630 2024-05-27T00:40:18.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.904+0000 7ff7ed7fa640 1 -- 172.21.15.111:0/2471452379 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7ff7e8059530 con 0x7ff7f00b3630 2024-05-27T00:40:18.905 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:18.905 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3282640688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3282640688},{"type":"v1","addr":"172.21.15.153:6837","nonce":3282640688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2471245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2471245114},{"type":"v1","addr":"172.21.15.111:6833","nonce":2471245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/543157056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":543157056},{"type":"v1","addr":"172.21.15.111:6835","nonce":543157056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:39:20.908117+0000","modified":"2024-05-27T00:39:37.972042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4320":{"gid":4320,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6836/2203621473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2203621473},{"type":"v1","addr":"172.21.15.153:6836","nonce":2203621473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4320,"qdb_cluster":[4320]},"id":1}]} 2024-05-27T00:40:18.906 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.906+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff7d006be60 msgr2=0x7ff7d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.906+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff7d006be60 0x7ff7d006e320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0034000 tx=0x7ff7e002f000 comp rx=0 tx=0).stop 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.906+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 msgr2=0x7ff7f01515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.906+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 0x7ff7f01515e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800d800 tx=0x7ff7e800dcc0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 shutdown_connections 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7ff7d006be60 0x7ff7d006e320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7f0162390 0x7ff7f00ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7ff7f00b77e0 0x7ff7f00aac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 --2- 172.21.15.111:0/2471452379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7f00b3630 0x7ff7f01515e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 >> 172.21.15.111:0/2471452379 conn(0x7ff7f00d42c0 msgr2=0x7ff7f01615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:18.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 shutdown_connections 2024-05-27T00:40:18.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:18.907+0000 7ff7f70bd640 1 -- 172.21.15.111:0/2471452379 wait complete. 2024-05-27T00:40:18.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:19.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.017+0000 7f764a83f640 1 Processor -- start 2024-05-27T00:40:19.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.017+0000 7f764a83f640 1 -- start start 2024-05-27T00:40:19.020 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f764a83f640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f764406b390 0x7f76440cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f764a83f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f76440726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f764a83f640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76440b36e0 con 0x7f7644144000 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f764a83f640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f76440b42f0 con 0x7f764406b390 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f764a83f640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f76440b39c0 con 0x7f76440cdc80 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f7643fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f7644144000 0x7f76440b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36168/0 (socket says 172.21.15.111:36168) 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f7643fff640 1 -- 172.21.15.111:0/1777989415 learned_addr learned my addr 172.21.15.111:0/1777989415 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083698788 0 0) 0x7f76440b36e0 con 0x7f7644144000 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f762c003520 con 0x7f7644144000 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f76440726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.018+0000 7f76437fe640 1 --2- 172.21.15.111:0/1777989415 >> v2:172.21.15.153:3300/0 conn(0x7f764406b390 0x7f76440cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3389675635 0 0) 0x7f762c003520 con 0x7f7644144000 2024-05-27T00:40:19.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 msgr2=0x7f76440726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f76440726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 >> v2:172.21.15.153:3300/0 conn(0x7f764406b390 msgr2=0x7f76440cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 --2- 172.21.15.111:0/1777989415 >> v2:172.21.15.153:3300/0 conn(0x7f764406b390 0x7f76440cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76440b3ca0 con 0x7f7644144000 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f7634001670 con 0x7f7644144000 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f76437fe640 1 --2- 172.21.15.111:0/1777989415 >> v2:172.21.15.153:3300/0 conn(0x7f764406b390 0x7f76440cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f76440726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 >> v1:172.21.15.111:6789/0 conn(0x7f7644144000 legacy=0x7f76440b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f762c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c003fa0 0x7f762c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f762c004d40 0x7f762c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f76440b42f0 con 0x7f762c004d40 2024-05-27T00:40:19.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76440b39c0 con 0x7f762c003fa0 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f762c009bb0 con 0x7f76440cdc80 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f762c004d40 0x7f762c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f76437fe640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c003fa0 0x7f762c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7643fff640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f762c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 -- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 msgr2=0x7f762c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f76440cdc80 0x7f762c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 -- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c003fa0 msgr2=0x7f762c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c003fa0 0x7f762c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.019+0000 7f7648ddc640 1 -- 172.21.15.111:0/1777989415 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c003520 con 0x7f762c004d40 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f762c004d40 0x7f762c0051a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f763c0073b0 tx=0x7f762c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7058bca9e479b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c032070 con 0x7f762c004d40 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763c003b90 con 0x7f762c004d40 2024-05-27T00:40:19.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f7642ffd640 1 -- 172.21.15.111:0/1777989415 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c009c80 con 0x7f762c004d40 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f764a83f640 1 -- 172.21.15.111:0/1777989415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f762c004d40 msgr2=0x7f762c0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f764a83f640 1 --2- 172.21.15.111:0/1777989415 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f762c004d40 0x7f762c0051a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f763c0073b0 tx=0x7f762c005830 comp rx=0 tx=0).stop 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.020+0000 7f7648ddc640 1 -- 172.21.15.111:0/1777989415 reap_dead start 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.021+0000 7f764a83f640 1 -- 172.21.15.111:0/1777989415 shutdown_connections 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.021+0000 7f764a83f640 1 -- 172.21.15.111:0/1777989415 >> 172.21.15.111:0/1777989415 conn(0x7f76440b1c00 msgr2=0x7f76440b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.021+0000 7f764a83f640 1 -- 172.21.15.111:0/1777989415 shutdown_connections 2024-05-27T00:40:19.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.021+0000 7f764a83f640 1 -- 172.21.15.111:0/1777989415 wait complete. 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.022+0000 7f764a83f640 1 Processor -- start 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.022+0000 7f764a83f640 1 -- start start 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c004d40 0x7f7644139130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 0x7f76441e0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f764413b490 0x7f764413b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7644072850 con 0x7f76441e02e0 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f76440726d0 con 0x7f762c004d40 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f76440729d0 con 0x7f764413b490 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 0x7f76441e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 0x7f76441e0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36644/0 (socket says 172.21.15.111:36644) 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 -- 172.21.15.111:0/1006686870 learned_addr learned my addr 172.21.15.111:0/1006686870 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f7643fff640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c004d40 0x7f7644139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f764413b490 0x7f764413b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 -- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f764413b490 msgr2=0x7f764413b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f764413b490 0x7f764413b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 -- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c004d40 msgr2=0x7f7644139130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c004d40 0x7f7644139130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 -- 172.21.15.111:0/1006686870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764413c130 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f7648ddc640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f764413b490 0x7f764413b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f76437fe640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 0x7f76441e0760 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f76380115e0 tx=0x7f7638011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800f040 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7638004230 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7644149ba0 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800f7b0 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.023+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764414a160 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.024+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7644068680 con 0x7f76441e02e0 2024-05-27T00:40:19.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.025+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f763800f950 con 0x7f76441e02e0 2024-05-27T00:40:19.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.025+0000 7f7640ff9640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f763006be60 0x7f763006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.025+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f763808b590 con 0x7f76441e02e0 2024-05-27T00:40:19.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.025+0000 7f7643fff640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f763006be60 0x7f763006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.026+0000 7f7643fff640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f763006be60 0x7f763006e320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7634007e70 tx=0x7f76340078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.029+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7638019020 con 0x7f76441e02e0 2024-05-27T00:40:19.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.203+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76440af930 con 0x7f76441e02e0 2024-05-27T00:40:19.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.204+0000 7f7640ff9640 1 -- 172.21.15.111:0/1006686870 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18996 (secure 0 0 0) 0x7f76380594d0 con 0x7f76441e02e0 2024-05-27T00:40:19.206 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:19.206 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":27,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:02.032592+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:39:18.903481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T00:39:19.886511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T00:39:23.607133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.15","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.206+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f763006be60 msgr2=0x7f763006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.206+0000 7f764a83f640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f763006be60 0x7f763006e320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7634007e70 tx=0x7f76340078f0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.206+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 msgr2=0x7f76441e0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.206+0000 7f764a83f640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 0x7f76441e0760 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f76380115e0 tx=0x7f7638011aa0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 shutdown_connections 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f763006be60 0x7f763006e320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f764413b490 0x7f764413b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76441e02e0 0x7f76441e0760 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 --2- 172.21.15.111:0/1006686870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f762c004d40 0x7f7644139130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 >> 172.21.15.111:0/1006686870 conn(0x7f76440b1c00 msgr2=0x7f76440b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 shutdown_connections 2024-05-27T00:40:19.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.207+0000 7f764a83f640 1 -- 172.21.15.111:0/1006686870 wait complete. 2024-05-27T00:40:19.219 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-27T00:40:19.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.317+0000 7f346845d640 1 Processor -- start 2024-05-27T00:40:19.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.317+0000 7f346845d640 1 -- start start 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.318+0000 7f346845d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34600b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.318+0000 7f346845d640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34600b77e0 0x7f3460161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.318+0000 7f346845d640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f346006cc70 con 0x7f3460162390 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.318+0000 7f346845d640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f34600d1e50 con 0x7f34600b77e0 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.318+0000 7f346845d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f34600d2130 con 0x7f34600b3630 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34600b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34659f8640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34600b77e0 0x7f3460161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34600b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:33164/0 (socket says 172.21.15.111:33164) 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34669fa640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3460162390 0x7f34600d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:40688/0 (socket says 172.21.15.111:40688) 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 -- 172.21.15.111:0/430960868 learned_addr learned my addr 172.21.15.111:0/430960868 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:19.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 -- 172.21.15.111:0/430960868 >> v2:172.21.15.153:3300/0 conn(0x7f34600b77e0 msgr2=0x7f3460161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 --2- 172.21.15.111:0/430960868 >> v2:172.21.15.153:3300/0 conn(0x7f34600b77e0 0x7f3460161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 -- 172.21.15.111:0/430960868 >> v1:172.21.15.153:6789/0 conn(0x7f3460162390 legacy=0x7f34600d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.319+0000 7f34661f9640 1 -- 172.21.15.111:0/430960868 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34600d2410 con 0x7f34600b3630 2024-05-27T00:40:19.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.320+0000 7f34661f9640 1 --2- 172.21.15.111:0/430960868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34600b72a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3450009e50 tx=0x7f345002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b45677ee60ae31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.320+0000 7f34651f7640 1 -- 172.21.15.111:0/430960868 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450036070 con 0x7f34600b3630 2024-05-27T00:40:19.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.320+0000 7f34651f7640 1 -- 172.21.15.111:0/430960868 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3450031600 con 0x7f34600b3630 2024-05-27T00:40:19.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.320+0000 7f34651f7640 1 -- 172.21.15.111:0/430960868 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500319d0 con 0x7f34600b3630 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.320+0000 7f346845d640 1 -- 172.21.15.111:0/430960868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 msgr2=0x7f34600b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.320+0000 7f346845d640 1 --2- 172.21.15.111:0/430960868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34600b72a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3450009e50 tx=0x7f345002a710 comp rx=0 tx=0).stop 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.321+0000 7f346845d640 1 -- 172.21.15.111:0/430960868 shutdown_connections 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.321+0000 7f346845d640 1 --2- 172.21.15.111:0/430960868 >> v2:172.21.15.153:3300/0 conn(0x7f34600b77e0 0x7f3460161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.321+0000 7f346845d640 1 --2- 172.21.15.111:0/430960868 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34600b72a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.321+0000 7f346845d640 1 -- 172.21.15.111:0/430960868 >> 172.21.15.111:0/430960868 conn(0x7f34600d42c0 msgr2=0x7f34600d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.321+0000 7f346845d640 1 -- 172.21.15.111:0/430960868 shutdown_connections 2024-05-27T00:40:19.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.321+0000 7f346845d640 1 -- 172.21.15.111:0/430960868 wait complete. 2024-05-27T00:40:19.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.322+0000 7f346845d640 1 Processor -- start 2024-05-27T00:40:19.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.322+0000 7f346845d640 1 -- start start 2024-05-27T00:40:19.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.322+0000 7f346845d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34601525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.322+0000 7f346845d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 0x7f34600afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.322+0000 7f346845d640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3460162390 0x7f34600b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.322+0000 7f346845d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34600d34a0 con 0x7f34600b77e0 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f346845d640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f34600d3320 con 0x7f3460162390 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f346845d640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f34600d3620 con 0x7f34600b3630 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 0x7f34600afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 0x7f34600afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36648/0 (socket says 172.21.15.111:36648) 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 -- 172.21.15.111:0/1590350322 learned_addr learned my addr 172.21.15.111:0/1590350322 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34661f9640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34601525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34669fa640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3460162390 0x7f34600b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 -- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 msgr2=0x7f34601525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34601525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 -- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3460162390 msgr2=0x7f34600b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3460162390 0x7f34600b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 -- 172.21.15.111:0/1590350322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34600b0000 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34669fa640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3460162390 0x7f34600b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34661f9640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34601525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f34659f8640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 0x7f34600afac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9b0 tx=0x7f345400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454017070 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345400c9a0 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34601f7ad0 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400cd90 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.323+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34601f8090 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.324+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34600b1560 con 0x7f34600b77e0 2024-05-27T00:40:19.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.325+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f345401a070 con 0x7f34600b77e0 2024-05-27T00:40:19.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.325+0000 7f345effd640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f344c06be30 0x7f344c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.325+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6764+0+0 (secure 0 0 0) 0x7f345408b600 con 0x7f34600b77e0 2024-05-27T00:40:19.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.326+0000 7f34661f9640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f344c06be30 0x7f344c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.326+0000 7f34661f9640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f344c06be30 0x7f344c06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3450033000 tx=0x7f3450009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.330+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3454059b90 con 0x7f34600b77e0 2024-05-27T00:40:19.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.537+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3460068680 con 0x7f34600b77e0 2024-05-27T00:40:19.539 INFO:tasks.ceph.mon.a.smithi111.stderr:2024-05-27T00:40:19.538+0000 7f45b5e7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T00:40:19.540 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.540+0000 7f345effd640 1 -- 172.21.15.111:0/1590350322 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f3454059490 con 0x7f34600b77e0 2024-05-27T00:40:19.540 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-27T00:40:19.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f344c06be30 msgr2=0x7f344c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f344c06be30 0x7f344c06e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3450033000 tx=0x7f3450009000 comp rx=0 tx=0).stop 2024-05-27T00:40:19.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 msgr2=0x7f34600afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 0x7f34600afac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9b0 tx=0x7f345400ee70 comp rx=0 tx=0).stop 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 shutdown_connections 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f344c06be30 0x7f344c06e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3460162390 0x7f34600b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34600b77e0 0x7f34600afac0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 --2- 172.21.15.111:0/1590350322 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f34600b3630 0x7f34601525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 >> 172.21.15.111:0/1590350322 conn(0x7f34600d42c0 msgr2=0x7f34600b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 shutdown_connections 2024-05-27T00:40:19.544 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.542+0000 7f346845d640 1 -- 172.21.15.111:0/1590350322 wait complete. 2024-05-27T00:40:19.547 INFO:tasks.ceph.mds.d.smithi153.stderr: -5319> 2024-05-27T00:39:25.475+0000 7f7a2c5b4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T00:40:19.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-27T00:40:19.580 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2024-05-27T00:40:19.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.657+0000 7fcc92e31640 1 Processor -- start 2024-05-27T00:40:19.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.657+0000 7fcc92e31640 1 -- start start 2024-05-27T00:40:19.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc92e31640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcc8c0b3630 0x7fcc8c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc92e31640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcc8c0b77e0 0x7fcc8c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc92e31640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c162390 0x7fcc8c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc92e31640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c06cc70 con 0x7fcc8c0b3630 2024-05-27T00:40:19.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc92e31640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c0d1e50 con 0x7fcc8c0b77e0 2024-05-27T00:40:19.661 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc92e31640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c0d2130 con 0x7fcc8c162390 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc90bcd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcc8c0b3630 0x7fcc8c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc90bcd640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fcc8c0b3630 0x7fcc8c0b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36656/0 (socket says 172.21.15.111:36656) 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.658+0000 7fcc90bcd640 1 -- 172.21.15.111:0/3770522062 learned_addr learned my addr 172.21.15.111:0/3770522062 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc913ce640 1 --2- 172.21.15.111:0/3770522062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c162390 0x7fcc8c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc8bfff640 1 --2- 172.21.15.111:0/3770522062 >> v2:172.21.15.153:3300/0 conn(0x7fcc8c0b77e0 0x7fcc8c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc90bcd640 1 -- 172.21.15.111:0/3770522062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c162390 msgr2=0x7fcc8c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc90bcd640 1 --2- 172.21.15.111:0/3770522062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c162390 0x7fcc8c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc90bcd640 1 -- 172.21.15.111:0/3770522062 >> v2:172.21.15.153:3300/0 conn(0x7fcc8c0b77e0 msgr2=0x7fcc8c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc90bcd640 1 --2- 172.21.15.111:0/3770522062 >> v2:172.21.15.153:3300/0 conn(0x7fcc8c0b77e0 0x7fcc8c161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.662 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.659+0000 7fcc90bcd640 1 -- 172.21.15.111:0/3770522062 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c0d2410 con 0x7fcc8c0b3630 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.660+0000 7fcc90bcd640 1 --2- 172.21.15.111:0/3770522062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c0b72a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c06d5d0 tx=0x7fcc7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5c25818fd479a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.660+0000 7fcc8b7fe640 1 -- 172.21.15.111:0/3770522062 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c036070 con 0x7fcc8c0b3630 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.660+0000 7fcc8b7fe640 1 -- 172.21.15.111:0/3770522062 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7c031600 con 0x7fcc8c0b3630 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.660+0000 7fcc8b7fe640 1 -- 172.21.15.111:0/3770522062 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0319d0 con 0x7fcc8c0b3630 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.661+0000 7fcc92e31640 1 -- 172.21.15.111:0/3770522062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c0b3630 msgr2=0x7fcc8c0b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.661+0000 7fcc92e31640 1 --2- 172.21.15.111:0/3770522062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c0b72a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c06d5d0 tx=0x7fcc7c02a6a0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 -- 172.21.15.111:0/3770522062 shutdown_connections 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 --2- 172.21.15.111:0/3770522062 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c162390 0x7fcc8c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 --2- 172.21.15.111:0/3770522062 >> v2:172.21.15.153:3300/0 conn(0x7fcc8c0b77e0 0x7fcc8c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 --2- 172.21.15.111:0/3770522062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c0b72a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 -- 172.21.15.111:0/3770522062 >> 172.21.15.111:0/3770522062 conn(0x7fcc8c0d42c0 msgr2=0x7fcc8c0d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 -- 172.21.15.111:0/3770522062 shutdown_connections 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.662+0000 7fcc92e31640 1 -- 172.21.15.111:0/3770522062 wait complete. 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.663+0000 7fcc92e31640 1 Processor -- start 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.663+0000 7fcc92e31640 1 -- start start 2024-05-27T00:40:19.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c0b77e0 0x7fcc8c1e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 0x7fcc8c1e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0d32d0 con 0x7fcc8c162390 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0d3150 con 0x7fcc8c0b3630 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c0d3450 con 0x7fcc8c0b77e0 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 0x7fcc8c1e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 0x7fcc8c1e9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36668/0 (socket says 172.21.15.111:36668) 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 -- 172.21.15.111:0/931422036 learned_addr learned my addr 172.21.15.111:0/931422036 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:19.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc90bcd640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc8bfff640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c0b77e0 0x7fcc8c1e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 -- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c0b77e0 msgr2=0x7fcc8c1e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c0b77e0 0x7fcc8c1e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 -- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc8c0b3630 msgr2=0x7fcc8c161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 -- 172.21.15.111:0/931422036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c0cda60 con 0x7fcc8c162390 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc913ce640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 0x7fcc8c1e9a00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d830 tx=0x7fcc8400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400b6c0 con 0x7fcc8c162390 2024-05-27T00:40:19.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc84019070 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400bac0 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.664+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c0cdd40 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.665+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c0ce300 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.666+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8c068680 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.666+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc8401e460 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.666+0000 7fcc897fa640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcc6c06be30 0x7fcc6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.667+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6850+0+0 (secure 0 0 0) 0x7fcc8408b780 con 0x7fcc8c162390 2024-05-27T00:40:19.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.667+0000 7fcc90bcd640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcc6c06be30 0x7fcc6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:19.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.667+0000 7fcc90bcd640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcc6c06be30 0x7fcc6c06e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c033000 tx=0x7fcc7c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:19.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.671+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcc840596f0 con 0x7fcc8c162390 2024-05-27T00:40:19.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:19.881+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc8c0cfc90 con 0x7fcc8c162390 2024-05-27T00:40:20.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.540+0000 7fcc897fa640 1 -- 172.21.15.111:0/931422036 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fcc8405d9f0 con 0x7fcc8c162390 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcc6c06be30 msgr2=0x7fcc6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcc6c06be30 0x7fcc6c06e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c033000 tx=0x7fcc7c009000 comp rx=0 tx=0).stop 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 msgr2=0x7fcc8c1e9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 0x7fcc8c1e9a00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400d830 tx=0x7fcc8400dcf0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 shutdown_connections 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fcc6c06be30 0x7fcc6c06e2f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc8c162390 0x7fcc8c1e9a00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fcc8c0b77e0 0x7fcc8c1e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 --2- 172.21.15.111:0/931422036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc8c0b3630 0x7fcc8c161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 >> 172.21.15.111:0/931422036 conn(0x7fcc8c0d42c0 msgr2=0x7fcc8c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:20.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 shutdown_connections 2024-05-27T00:40:20.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.543+0000 7fcc92e31640 1 -- 172.21.15.111:0/931422036 wait complete. 2024-05-27T00:40:20.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-27T00:40:20.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.663+0000 7f878e3d5640 1 Processor -- start 2024-05-27T00:40:20.665 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.663+0000 7f878e3d5640 1 -- start start 2024-05-27T00:40:20.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878e3d5640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f87880d3440 0x7f87880b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:20.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878e3d5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f87880b34d0 0x7f878806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:20.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878e3d5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878806b840 con 0x7f8788161c10 2024-05-27T00:40:20.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878e3d5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f878806bb20 con 0x7f87880b34d0 2024-05-27T00:40:20.666 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878e3d5640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f878806be00 con 0x7f87880d3440 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878c972640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f87880b34d0 0x7f878806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f8787fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8788161c10 0x7f87880d2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56036/0 (socket says 172.21.15.111:56036) 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f8787fff640 1 -- 172.21.15.111:0/2890926403 learned_addr learned my addr 172.21.15.111:0/2890926403 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878c972640 1 -- 172.21.15.111:0/2890926403 >> v2:172.21.15.153:3300/0 conn(0x7f87880d3440 msgr2=0x7f87880b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878c972640 1 --2- 172.21.15.111:0/2890926403 >> v2:172.21.15.153:3300/0 conn(0x7f87880d3440 0x7f87880b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878c972640 1 -- 172.21.15.111:0/2890926403 >> v1:172.21.15.111:6789/0 conn(0x7f8788161c10 legacy=0x7f87880d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f87877fe640 1 -- 172.21.15.111:0/2890926403 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3214108771 0 0) 0x7f878806b840 con 0x7f8788161c10 2024-05-27T00:40:20.667 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.664+0000 7f878c972640 1 -- 172.21.15.111:0/2890926403 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878806c0e0 con 0x7f87880b34d0 2024-05-27T00:40:20.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.665+0000 7f878c972640 1 --2- 172.21.15.111:0/2890926403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f878806c8e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f878808cb90 tx=0x7f878002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e70ff5d9f62d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:20.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.665+0000 7f87877fe640 1 -- 172.21.15.111:0/2890926403 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780007740 con 0x7f87880b34d0 2024-05-27T00:40:20.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.665+0000 7f87877fe640 1 -- 172.21.15.111:0/2890926403 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87800078e0 con 0x7f87880b34d0 2024-05-27T00:40:20.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.665+0000 7f87877fe640 1 -- 172.21.15.111:0/2890926403 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780007cb0 con 0x7f87880b34d0 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 -- 172.21.15.111:0/2890926403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 msgr2=0x7f878806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 --2- 172.21.15.111:0/2890926403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f878806c8e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f878808cb90 tx=0x7f878002a680 comp rx=0 tx=0).stop 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 -- 172.21.15.111:0/2890926403 shutdown_connections 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 --2- 172.21.15.111:0/2890926403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f878806c8e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 --2- 172.21.15.111:0/2890926403 >> v2:172.21.15.153:3300/0 conn(0x7f87880d3440 0x7f87880b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 -- 172.21.15.111:0/2890926403 >> 172.21.15.111:0/2890926403 conn(0x7f87880b5c40 msgr2=0x7f87880b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 -- 172.21.15.111:0/2890926403 shutdown_connections 2024-05-27T00:40:20.669 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.666+0000 7f878e3d5640 1 -- 172.21.15.111:0/2890926403 wait complete. 2024-05-27T00:40:20.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.667+0000 7f878e3d5640 1 Processor -- start 2024-05-27T00:40:20.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.667+0000 7f878e3d5640 1 -- start start 2024-05-27T00:40:20.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f8788151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:20.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87880d3440 0x7f87880cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:20.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8788161c10 0x7f878815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d0830 con 0x7f87880b34d0 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d06b0 con 0x7f8788161c10 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f87880b7340 con 0x7f87880d3440 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f8788151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f8788151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40178/0 (socket says 172.21.15.111:40178) 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 -- 172.21.15.111:0/4110557527 learned_addr learned my addr 172.21.15.111:0/4110557527 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 -- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87880d3440 msgr2=0x7f87880cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87880d3440 0x7f87880cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 -- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8788161c10 msgr2=0x7f878815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8788161c10 0x7f878815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 -- 172.21.15.111:0/4110557527 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87880b7d40 con 0x7f87880b34d0 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f8787fff640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f8788151550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f877800c430 tx=0x7f877800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780074e0 con 0x7f87880b34d0 2024-05-27T00:40:20.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87780047e0 con 0x7f87880b34d0 2024-05-27T00:40:20.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.668+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878815af50 con 0x7f87880b34d0 2024-05-27T00:40:20.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.669+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778004bd0 con 0x7f87880b34d0 2024-05-27T00:40:20.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.669+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878815b510 con 0x7f87880b34d0 2024-05-27T00:40:20.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.669+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8788068680 con 0x7f87880b34d0 2024-05-27T00:40:20.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.670+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8778011070 con 0x7f87880b34d0 2024-05-27T00:40:20.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.670+0000 7f87857fa640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f876806be60 0x7f876806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:20.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.670+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6850+0+0 (secure 0 0 0) 0x7f877808af30 con 0x7f87880b34d0 2024-05-27T00:40:20.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.671+0000 7f877ffff640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f876806be60 0x7f876806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:20.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.671+0000 7f877ffff640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f876806be60 0x7f876806e320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f878815fce0 tx=0x7f8770006c90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:20.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.675+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f877800e280 con 0x7f87880b34d0 2024-05-27T00:40:20.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:20.851+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87880d2630 con 0x7f87880b34d0 2024-05-27T00:40:21.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.544+0000 7f87857fa640 1 -- 172.21.15.111:0/4110557527 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f8778059460 con 0x7f87880b34d0 2024-05-27T00:40:21.547 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.550+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f876806be60 msgr2=0x7f876806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.550+0000 7f878e3d5640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f876806be60 0x7f876806e320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f878815fce0 tx=0x7f8770006c90 comp rx=0 tx=0).stop 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.550+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 msgr2=0x7f8788151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.550+0000 7f878e3d5640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f8788151550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f877800c430 tx=0x7f877800c8f0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 shutdown_connections 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f876806be60 0x7f876806e320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8788161c10 0x7f878815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f87880d3440 0x7f87880cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 --2- 172.21.15.111:0/4110557527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87880b34d0 0x7f8788151550 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 >> 172.21.15.111:0/4110557527 conn(0x7f87880b5c40 msgr2=0x7f87880d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:21.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.551+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 shutdown_connections 2024-05-27T00:40:21.554 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.552+0000 7f878e3d5640 1 -- 172.21.15.111:0/4110557527 wait complete. 2024-05-27T00:40:21.567 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-27T00:40:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.673+0000 7f466f57e640 1 Processor -- start 2024-05-27T00:40:21.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.674+0000 7f466f57e640 1 -- start start 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466f57e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f466813f340 0x7f46680cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466f57e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46680cce00 0x7f4668141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466f57e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4668142650 con 0x7f466813ef60 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466f57e640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f46680cd270 con 0x7f46680cce00 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466f57e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f46680cf210 con 0x7f466813f340 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46680cce00 0x7f4668141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46680cce00 0x7f4668141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40190/0 (socket says 172.21.15.111:40190) 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 -- 172.21.15.111:0/1736725961 learned_addr learned my addr 172.21.15.111:0/1736725961 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:21.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 -- 172.21.15.111:0/1736725961 >> v2:172.21.15.153:3300/0 conn(0x7f466813f340 msgr2=0x7f46680cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 --2- 172.21.15.111:0/1736725961 >> v2:172.21.15.153:3300/0 conn(0x7f466813f340 0x7f46680cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 -- 172.21.15.111:0/1736725961 >> v1:172.21.15.153:6789/0 conn(0x7f466813ef60 legacy=0x7f46680cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:21.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.675+0000 7f466db1b640 1 -- 172.21.15.111:0/1736725961 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46680cebc0 con 0x7f46680cce00 2024-05-27T00:40:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466db1b640 1 --2- 172.21.15.111:0/1736725961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46680cce00 0x7f4668141920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4668145e70 tx=0x7f466002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747c7f85613fece server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f465ffff640 1 -- 172.21.15.111:0/1736725961 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660036070 con 0x7f46680cce00 2024-05-27T00:40:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f465ffff640 1 -- 172.21.15.111:0/1736725961 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4660031600 con 0x7f46680cce00 2024-05-27T00:40:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f465ffff640 1 -- 172.21.15.111:0/1736725961 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600319d0 con 0x7f46680cce00 2024-05-27T00:40:21.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 -- 172.21.15.111:0/1736725961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46680cce00 msgr2=0x7f4668141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 --2- 172.21.15.111:0/1736725961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46680cce00 0x7f4668141920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4668145e70 tx=0x7f466002a6f0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 -- 172.21.15.111:0/1736725961 shutdown_connections 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 --2- 172.21.15.111:0/1736725961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46680cce00 0x7f4668141920 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 --2- 172.21.15.111:0/1736725961 >> v2:172.21.15.153:3300/0 conn(0x7f466813f340 0x7f46680cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 -- 172.21.15.111:0/1736725961 >> 172.21.15.111:0/1736725961 conn(0x7f46680b1c00 msgr2=0x7f46680b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 -- 172.21.15.111:0/1736725961 shutdown_connections 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.676+0000 7f466f57e640 1 -- 172.21.15.111:0/1736725961 wait complete. 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.677+0000 7f466f57e640 1 Processor -- start 2024-05-27T00:40:21.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 -- start start 2024-05-27T00:40:21.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46680cce00 0x7f46681429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 0x7f466813e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f466813f340 0x7f4668140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4668141e60 con 0x7f466813ef60 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f46680ce890 con 0x7f466813f340 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466f57e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4668141fe0 con 0x7f46680cce00 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466cb19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 0x7f466813e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466cb19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 0x7f466813e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40204/0 (socket says 172.21.15.111:40204) 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.678+0000 7f466cb19640 1 -- 172.21.15.111:0/3800682036 learned_addr learned my addr 172.21.15.111:0/3800682036 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466db1b640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f466813f340 0x7f4668140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466d31a640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46680cce00 0x7f46681429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466cb19640 1 -- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46680cce00 msgr2=0x7f46681429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:21.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466cb19640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46680cce00 0x7f46681429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466cb19640 1 -- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f466813f340 msgr2=0x7f4668140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466cb19640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f466813f340 0x7f4668140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466cb19640 1 -- 172.21.15.111:0/3800682036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46681f52b0 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466db1b640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f466813f340 0x7f4668140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466cb19640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 0x7f466813e7d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f465800b410 tx=0x7f465800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658015020 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658011850 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46681f5590 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658011c40 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.679+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46681f5b50 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.680+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4668068680 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.681+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4658018020 con 0x7f466813ef60 2024-05-27T00:40:21.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.681+0000 7f465dffb640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f464c06be30 0x7f464c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.681+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6412+0+0 (secure 0 0 0) 0x7f465808b060 con 0x7f466813ef60 2024-05-27T00:40:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.681+0000 7f466d31a640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f464c06be30 0x7f464c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:21.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.682+0000 7f466d31a640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f464c06be30 0x7f464c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4664007fd0 tx=0x7f4664009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:21.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.685+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f465801f020 con 0x7f466813ef60 2024-05-27T00:40:21.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:21.862+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46680b21f0 con 0x7f466813ef60 2024-05-27T00:40:22.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.549+0000 7f465dffb640 1 -- 172.21.15.111:0/3800682036 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f46580172c0 con 0x7f466813ef60 2024-05-27T00:40:22.552 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f464c06be30 msgr2=0x7f464c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f464c06be30 0x7f464c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4664007fd0 tx=0x7f4664009040 comp rx=0 tx=0).stop 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 msgr2=0x7f466813e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 0x7f466813e7d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f465800b410 tx=0x7f465800b8d0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 shutdown_connections 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f464c06be30 0x7f464c06e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f466813f340 0x7f4668140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f466813ef60 0x7f466813e7d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 --2- 172.21.15.111:0/3800682036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f46680cce00 0x7f46681429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 >> 172.21.15.111:0/3800682036 conn(0x7f46680b1c00 msgr2=0x7f4668160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:22.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 shutdown_connections 2024-05-27T00:40:22.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.553+0000 7f466f57e640 1 -- 172.21.15.111:0/3800682036 wait complete. 2024-05-27T00:40:22.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-27T00:40:22.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.674+0000 7fe017122640 1 Processor -- start 2024-05-27T00:40:22.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.674+0000 7fe017122640 1 -- start start 2024-05-27T00:40:22.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe017122640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe01013ef60 0x7fe0100cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:22.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe017122640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe0100ccfd0 0x7fe010141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:22.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe017122640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe010142650 con 0x7fe01013f340 2024-05-27T00:40:22.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe017122640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100cf210 con 0x7fe0100ccfd0 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe017122640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe0100cebc0 con 0x7fe01013ef60 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe007fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe01013f340 0x7fe0100cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56042/0 (socket says 172.21.15.111:56042) 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe0156bf640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe0100ccfd0 0x7fe010141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe007fff640 1 -- 172.21.15.111:0/2087822912 learned_addr learned my addr 172.21.15.111:0/2087822912 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe014ebe640 1 --2- 172.21.15.111:0/2087822912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe01013ef60 0x7fe0100cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe0156bf640 1 -- 172.21.15.111:0/2087822912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe01013ef60 msgr2=0x7fe0100cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe0156bf640 1 --2- 172.21.15.111:0/2087822912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe01013ef60 0x7fe0100cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe0156bf640 1 -- 172.21.15.111:0/2087822912 >> v1:172.21.15.111:6789/0 conn(0x7fe01013f340 legacy=0x7fe0100cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe0077fe640 1 -- 172.21.15.111:0/2087822912 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499437269 0 0) 0x7fe010142650 con 0x7fe01013f340 2024-05-27T00:40:22.678 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.675+0000 7fe0156bf640 1 -- 172.21.15.111:0/2087822912 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100ceea0 con 0x7fe0100ccfd0 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.676+0000 7fe0156bf640 1 --2- 172.21.15.111:0/2087822912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0100ccfd0 0x7fe010141920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a0f681e0afc3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.676+0000 7fe0077fe640 1 -- 172.21.15.111:0/2087822912 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008007740 con 0x7fe0100ccfd0 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.676+0000 7fe0077fe640 1 -- 172.21.15.111:0/2087822912 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0080078e0 con 0x7fe0100ccfd0 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.676+0000 7fe0077fe640 1 -- 172.21.15.111:0/2087822912 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008007cb0 con 0x7fe0100ccfd0 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 -- 172.21.15.111:0/2087822912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0100ccfd0 msgr2=0x7fe010141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 --2- 172.21.15.111:0/2087822912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0100ccfd0 0x7fe010141920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00802a660 comp rx=0 tx=0).stop 2024-05-27T00:40:22.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 -- 172.21.15.111:0/2087822912 shutdown_connections 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 --2- 172.21.15.111:0/2087822912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0100ccfd0 0x7fe010141920 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 --2- 172.21.15.111:0/2087822912 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe01013ef60 0x7fe0100cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 -- 172.21.15.111:0/2087822912 >> 172.21.15.111:0/2087822912 conn(0x7fe0100b1c00 msgr2=0x7fe0100b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 -- 172.21.15.111:0/2087822912 shutdown_connections 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.677+0000 7fe017122640 1 -- 172.21.15.111:0/2087822912 wait complete. 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.678+0000 7fe017122640 1 Processor -- start 2024-05-27T00:40:22.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.678+0000 7fe017122640 1 -- start start 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100ccfd0 0x7fe0101ef000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 0x7fe010156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01013f340 0x7fe01015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe010141e60 con 0x7fe01013ef60 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100ce890 con 0x7fe01013f340 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fe010141fe0 con 0x7fe0100ccfd0 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 0x7fe010156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 0x7fe010156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40222/0 (socket says 172.21.15.111:40222) 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 -- 172.21.15.111:0/1233301099 learned_addr learned my addr 172.21.15.111:0/1233301099 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe014ebe640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100ccfd0 0x7fe0101ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe0156bf640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01013f340 0x7fe01015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:22.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 -- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100ccfd0 msgr2=0x7fe0101ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100ccfd0 0x7fe0101ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 -- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01013f340 msgr2=0x7fe01015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01013f340 0x7fe01015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 -- 172.21.15.111:0/1233301099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe010145210 con 0x7fe01013ef60 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe014ebe640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100ccfd0 0x7fe0101ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe0156bf640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01013f340 0x7fe01015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe007fff640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 0x7fe010156840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c460 tx=0x7fe00000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000018070 con 0x7fe01013ef60 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe000007500 con 0x7fe01013ef60 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0101454f0 con 0x7fe01013ef60 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000004a20 con 0x7fe01013ef60 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.679+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01015c500 con 0x7fe01013ef60 2024-05-27T00:40:22.685 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.680+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0100cdeb0 con 0x7fe01013ef60 2024-05-27T00:40:22.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.681+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe000004bc0 con 0x7fe01013ef60 2024-05-27T00:40:22.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.681+0000 7fe0057fa640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdffc06be30 0x7fdffc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:22.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.681+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7fe00008a230 con 0x7fe01013ef60 2024-05-27T00:40:22.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.682+0000 7fe014ebe640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdffc06be30 0x7fdffc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:22.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.682+0000 7fe014ebe640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdffc06be30 0x7fdffc06e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe0101459a0 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:22.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.686+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe0100cdeb0 con 0x7fe01013ef60 2024-05-27T00:40:22.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:22.859+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe0101580c0 con 0x7fe01013ef60 2024-05-27T00:40:23.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.558+0000 7fe0057fa640 1 -- 172.21.15.111:0/1233301099 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7fe000058b60 con 0x7fe01013ef60 2024-05-27T00:40:23.560 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data_ec' removed 2024-05-27T00:40:23.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdffc06be30 msgr2=0x7fdffc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:23.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdffc06be30 0x7fdffc06e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe0101459a0 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-05-27T00:40:23.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 msgr2=0x7fe010156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:23.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 0x7fe010156840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c460 tx=0x7fe00000c920 comp rx=0 tx=0).stop 2024-05-27T00:40:23.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 shutdown_connections 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdffc06be30 0x7fdffc06e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01013f340 0x7fe01015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01013ef60 0x7fe010156840 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 --2- 172.21.15.111:0/1233301099 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fe0100ccfd0 0x7fe0101ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 >> 172.21.15.111:0/1233301099 conn(0x7fe0100b1c00 msgr2=0x7fe0100b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 shutdown_connections 2024-05-27T00:40:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.560+0000 7fe017122640 1 -- 172.21.15.111:0/1233301099 wait complete. 2024-05-27T00:40:23.580 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-27T00:40:23.581 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-27T00:40:23.581 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-27T00:40:23.581 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T00:40:23.581 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-27T00:40:23.581 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-27T00:40:23.581 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-27T00:40:23.581 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T00:40:23.582 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-27T00:40:23.582 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-27T00:40:23.582 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-27T00:40:23.582 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T00:40:23.582 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-27T00:40:23.582 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-27T00:40:23.583 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-27T00:40:23.583 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T00:40:23.591 INFO:tasks.ceph.mds.a:Stopped 2024-05-27T00:40:23.592 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-27T00:40:23.593 INFO:tasks.ceph.mds.c:Stopped 2024-05-27T00:40:23.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-27T00:40:23.596 INFO:tasks.ceph.mds.b:Stopped 2024-05-27T00:40:23.596 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-27T00:40:23.597 INFO:tasks.ceph.mds.d:Stopped 2024-05-27T00:40:23.597 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-27T00:40:23.635 INFO:tasks.ceph.mds.a:Started 2024-05-27T00:40:23.635 DEBUG:teuthology.parallel:result is None 2024-05-27T00:40:23.636 INFO:tasks.ceph.mds.c:Started 2024-05-27T00:40:23.636 DEBUG:teuthology.parallel:result is None 2024-05-27T00:40:23.637 INFO:tasks.ceph.mds.b:Started 2024-05-27T00:40:23.638 DEBUG:teuthology.parallel:result is None 2024-05-27T00:40:23.638 INFO:tasks.ceph.mds.d:Started 2024-05-27T00:40:23.638 DEBUG:teuthology.parallel:result is None 2024-05-27T00:40:23.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-27T00:40:23.713 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2024-05-27T00:40:23.713 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2024-05-27T00:40:23.721 INFO:tasks.ceph.mds.b.smithi153.stdout:starting mds.b at 2024-05-27T00:40:23.721 INFO:tasks.ceph.mds.d.smithi153.stdout:starting mds.d at 2024-05-27T00:40:23.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.743+0000 7f3773e10640 1 Processor -- start 2024-05-27T00:40:23.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.743+0000 7f3773e10640 1 -- start start 2024-05-27T00:40:23.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3773e10640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b3630 0x7f376c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3773e10640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f376c0b77e0 0x7f376c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3773e10640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f376c162390 0x7f376c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3773e10640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f376c06cc70 con 0x7f376c0b77e0 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3773e10640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f376c0d1e50 con 0x7f376c162390 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3773e10640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f376c0d2130 con 0x7f376c0b3630 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f37713ab640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f376c0b77e0 0x7f376c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f37713ab640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f376c0b77e0 0x7f376c161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40254/0 (socket says 172.21.15.111:40254) 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f37713ab640 1 -- 172.21.15.111:0/2594637511 learned_addr learned my addr 172.21.15.111:0/2594637511 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f3771bac640 1 --2- 172.21.15.111:0/2594637511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b3630 0x7f376c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.744+0000 7f37723ad640 1 --2- 172.21.15.111:0/2594637511 >> v2:172.21.15.153:3300/0 conn(0x7f376c162390 0x7f376c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.745+0000 7f37713ab640 1 -- 172.21.15.111:0/2594637511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b3630 msgr2=0x7f376c0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.745+0000 7f37713ab640 1 --2- 172.21.15.111:0/2594637511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b3630 0x7f376c0b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.745+0000 7f37713ab640 1 -- 172.21.15.111:0/2594637511 >> v2:172.21.15.153:3300/0 conn(0x7f376c162390 msgr2=0x7f376c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.745+0000 7f37713ab640 1 --2- 172.21.15.111:0/2594637511 >> v2:172.21.15.153:3300/0 conn(0x7f376c162390 0x7f376c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.745+0000 7f37713ab640 1 -- 172.21.15.111:0/2594637511 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c0d2410 con 0x7f376c0b77e0 2024-05-27T00:40:23.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.745+0000 7f37713ab640 1 --2- 172.21.15.111:0/2594637511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b77e0 0x7f376c161e50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f376c0d3c00 tx=0x7f375c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c08d9820217f06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:23.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3770baa640 1 -- 172.21.15.111:0/2594637511 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c037070 con 0x7f376c0b77e0 2024-05-27T00:40:23.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3770baa640 1 -- 172.21.15.111:0/2594637511 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c032440 con 0x7f376c0b77e0 2024-05-27T00:40:23.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3770baa640 1 -- 172.21.15.111:0/2594637511 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c032810 con 0x7f376c0b77e0 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 -- 172.21.15.111:0/2594637511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b77e0 msgr2=0x7f376c161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 --2- 172.21.15.111:0/2594637511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b77e0 0x7f376c161e50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f376c0d3c00 tx=0x7f375c02a1e0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 -- 172.21.15.111:0/2594637511 shutdown_connections 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 --2- 172.21.15.111:0/2594637511 >> v2:172.21.15.153:3300/0 conn(0x7f376c162390 0x7f376c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 --2- 172.21.15.111:0/2594637511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b77e0 0x7f376c161e50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 --2- 172.21.15.111:0/2594637511 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b3630 0x7f376c0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 -- 172.21.15.111:0/2594637511 >> 172.21.15.111:0/2594637511 conn(0x7f376c0d42c0 msgr2=0x7f376c0d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 -- 172.21.15.111:0/2594637511 shutdown_connections 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.746+0000 7f3773e10640 1 -- 172.21.15.111:0/2594637511 wait complete. 2024-05-27T00:40:23.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.747+0000 7f3773e10640 1 Processor -- start 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.747+0000 7f3773e10640 1 -- start start 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3773e10640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 0x7f376c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3773e10640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b77e0 0x7f376c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3773e10640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f376c162390 0x7f376c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3773e10640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f376c0d32d0 con 0x7f376c0b3630 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3773e10640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f376c0d3150 con 0x7f376c162390 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3773e10640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f376c0d3450 con 0x7f376c0b77e0 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 0x7f376c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 0x7f376c15c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40264/0 (socket says 172.21.15.111:40264) 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 -- 172.21.15.111:0/2063768759 learned_addr learned my addr 172.21.15.111:0/2063768759 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f37713ab640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b77e0 0x7f376c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f37723ad640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f376c162390 0x7f376c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 -- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b77e0 msgr2=0x7f376c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b77e0 0x7f376c15ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 -- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f376c162390 msgr2=0x7f376c156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f376c162390 0x7f376c156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f3771bac640 1 -- 172.21.15.111:0/2063768759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c158a70 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.748+0000 7f37723ad640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f376c162390 0x7f376c156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.749+0000 7f3771bac640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 0x7f376c15c6e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f376800eed0 tx=0x7f376800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.749+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768018020 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.749+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3768014400 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.749+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680147d0 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.749+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c158d50 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.749+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c0cc360 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.750+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3768005d10 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.750+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c068680 con 0x7f376c0b3630 2024-05-27T00:40:23.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.751+0000 7f37627fc640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375006be30 0x7f375006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:23.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.751+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f376808b420 con 0x7f376c0b3630 2024-05-27T00:40:23.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.751+0000 7f37713ab640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375006be30 0x7f375006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:23.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.751+0000 7f37713ab640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375006be30 0x7f375006e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f375c034000 tx=0x7f375c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:23.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.756+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3768022070 con 0x7f376c0b3630 2024-05-27T00:40:23.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:23.950+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f376c0cef10 con 0x7f376c0b3630 2024-05-27T00:40:24.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.568+0000 7f37627fc640 1 -- 172.21.15.111:0/2063768759 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f3768059f40 con 0x7f376c0b3630 2024-05-27T00:40:24.571 INFO:teuthology.orchestra.run.smithi111.stderr: removed all blocklist entries 2024-05-27T00:40:24.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375006be30 msgr2=0x7f375006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375006be30 0x7f375006e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f375c034000 tx=0x7f375c02e000 comp rx=0 tx=0).stop 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 msgr2=0x7f376c15c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 0x7f376c15c6e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f376800eed0 tx=0x7f376800b480 comp rx=0 tx=0).stop 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 shutdown_connections 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375006be30 0x7f375006e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f376c162390 0x7f376c156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f376c0b77e0 0x7f376c15ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 --2- 172.21.15.111:0/2063768759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f376c0b3630 0x7f376c15c6e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 >> 172.21.15.111:0/2063768759 conn(0x7f376c0d42c0 msgr2=0x7f376c0b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 shutdown_connections 2024-05-27T00:40:24.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.572+0000 7f3773e10640 1 -- 172.21.15.111:0/2063768759 wait complete. 2024-05-27T00:40:24.588 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-27T00:40:24.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.688+0000 7f9b92a58640 1 Processor -- start 2024-05-27T00:40:24.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.688+0000 7f9b92a58640 1 -- start start 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b92a58640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9b8c144000 0x7f9b8c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b92a58640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 0x7f9b8c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b92a58640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c0b3440 con 0x7f9b8c0cdc80 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b92a58640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c0b42f0 con 0x7f9b8c144000 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b92a58640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c0b36e0 con 0x7f9b8c06b390 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b90ff5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9b8c0cdc80 0x7f9b8c0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56068/0 (socket says 172.21.15.111:56068) 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b90ff5640 1 -- 172.21.15.111:0/504157426 learned_addr learned my addr 172.21.15.111:0/504157426 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b8bfff640 1 --2- 172.21.15.111:0/504157426 >> v2:172.21.15.153:3300/0 conn(0x7f9b8c144000 0x7f9b8c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.689+0000 7f9b8b7fe640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 0x7f9b8c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845500759 0 0) 0x7f9b8c0b3440 con 0x7f9b8c0cdc80 2024-05-27T00:40:24.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b74003520 con 0x7f9b8c0cdc80 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4213497712 0 0) 0x7f9b74003520 con 0x7f9b8c0cdc80 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 msgr2=0x7f9b8c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 0x7f9b8c0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 >> v2:172.21.15.153:3300/0 conn(0x7f9b8c144000 msgr2=0x7f9b8c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 --2- 172.21.15.111:0/504157426 >> v2:172.21.15.153:3300/0 conn(0x7f9b8c144000 0x7f9b8c0b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c0b39c0 con 0x7f9b8c0cdc80 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f9b840019f0 con 0x7f9b8c0cdc80 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 >> v1:172.21.15.111:6789/0 conn(0x7f9b8c0cdc80 legacy=0x7f9b8c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 0x7f9b740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b74003f30 0x7f9b74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0b42f0 con 0x7f9b74004d40 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0b36e0 con 0x7f9b74003f30 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7400b9c0 con 0x7f9b8c06b390 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b90ff5640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8bfff640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 0x7f9b740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b8b7fe640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b74003f30 0x7f9b74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b90ff5640 1 -- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 msgr2=0x7f9b740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b90ff5640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c06b390 0x7f9b740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b90ff5640 1 -- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b74003f30 msgr2=0x7f9b74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b90ff5640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b74003f30 0x7f9b74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.690+0000 7f9b90ff5640 1 -- 172.21.15.111:0/504157426 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b74003520 con 0x7f9b74004d40 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.691+0000 7f9b90ff5640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b740051c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0741a0 tx=0x7f9b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87589224f2658412 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.691+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400a4c0 con 0x7f9b74004d40 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.691+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8402f070 con 0x7f9b74004d40 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.691+0000 7f9b8affd640 1 -- 172.21.15.111:0/504157426 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8402bc60 con 0x7f9b74004d40 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b92a58640 1 -- 172.21.15.111:0/504157426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 msgr2=0x7f9b740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b92a58640 1 --2- 172.21.15.111:0/504157426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b740051c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0741a0 tx=0x7f9b74005830 comp rx=0 tx=0).stop 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b90ff5640 1 -- 172.21.15.111:0/504157426 reap_dead start 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b92a58640 1 -- 172.21.15.111:0/504157426 shutdown_connections 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b92a58640 1 -- 172.21.15.111:0/504157426 >> 172.21.15.111:0/504157426 conn(0x7f9b8c0b1c00 msgr2=0x7f9b8c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b92a58640 1 -- 172.21.15.111:0/504157426 shutdown_connections 2024-05-27T00:40:24.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.692+0000 7f9b92a58640 1 -- 172.21.15.111:0/504157426 wait complete. 2024-05-27T00:40:24.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.693+0000 7f9b92a58640 1 Processor -- start 2024-05-27T00:40:24.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.693+0000 7f9b92a58640 1 -- start start 2024-05-27T00:40:24.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b8c159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b8c15dd90 0x7f9b8c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c13ae80 0x7f9b8c13b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c072de0 con 0x7f9b74004d40 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0b3f80 con 0x7f9b8c15dd90 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c072f60 con 0x7f9b8c13ae80 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b8c159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b8c159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40286/0 (socket says 172.21.15.111:40286) 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 -- 172.21.15.111:0/3862330906 learned_addr learned my addr 172.21.15.111:0/3862330906 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8b7fe640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b8c15dd90 0x7f9b8c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b90ff5640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c13ae80 0x7f9b8c13b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 -- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c13ae80 msgr2=0x7f9b8c13b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c13ae80 0x7f9b8c13b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 -- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b8c15dd90 msgr2=0x7f9b8c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b8c15dd90 0x7f9b8c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 -- 172.21.15.111:0/3862330906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c159aa0 con 0x7f9b74004d40 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8b7fe640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b8c15dd90 0x7f9b8c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b8bfff640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b8c159560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c011ad0 tx=0x7f9b7c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00fbb0 con 0x7f9b74004d40 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c00fd50 con 0x7f9b74004d40 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c1329d0 con 0x7f9b74004d40 2024-05-27T00:40:24.702 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.694+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c014660 con 0x7f9b74004d40 2024-05-27T00:40:24.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.695+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c159ff0 con 0x7f9b74004d40 2024-05-27T00:40:24.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.695+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c068680 con 0x7f9b74004d40 2024-05-27T00:40:24.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.696+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b7c012040 con 0x7f9b74004d40 2024-05-27T00:40:24.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.696+0000 7f9b88ff9640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9b7806be60 0x7f9b7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:24.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.696+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9b7c08b360 con 0x7f9b74004d40 2024-05-27T00:40:24.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.697+0000 7f9b8b7fe640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9b7806be60 0x7f9b7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:24.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.697+0000 7f9b8b7fe640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9b7806be60 0x7f9b7806e320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c06b840 tx=0x7f9b80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:24.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.702+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9b7c019030 con 0x7f9b74004d40 2024-05-27T00:40:24.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.942+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9b8c0b4dd0 con 0x7f9b74004d40 2024-05-27T00:40:24.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.942+0000 7f9b88ff9640 1 -- 172.21.15.111:0/3862330906 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) ==== 80+0+6645 (secure 0 0 0) 0x7f9b7c059910 con 0x7f9b74004d40 2024-05-27T00:40:24.943 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "auth_dump": [ 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.a", 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD51VNmIrCSORAAVGHebu/KHoXn4GoUa82ycQ==", 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-05-27T00:40:24.944 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.b", 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNmCY2lCRAAnMGZi3U691ENXRTH5ZYfsw==", 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.c", 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNm57f1BBAAB+W3q8KIOIrG8mA9NSa8pg==", 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.946 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.d", 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNmZ6KqEBAA/W6j7dzA/RwaLjEYDywh4g==", 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.0", 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD91VNm5ZWHLxAA6mGY/B0YPVqLltiybJPeGQ==", 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.1", 2024-05-27T00:40:24.948 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAA1lNmGcaULRAAUiSBUVnQ5ceztu1ynUW/hg==", 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.2", 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAD1lNmBPWFKxAAgSMks9rTqSJjGBPXeixN6A==", 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.949 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.3", 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAG1lNm5zdBKRAAxZOTN3UsAoGjOM/SmjyTCw==", 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.4", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAM1lNmi6SGLhAAev9uBjq1OO/SOEC7j84yqg==", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.5", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAP1lNmDsrgLBAAvaAx28KGq+5CFEIu9l3lKg==", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.951 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.6", 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAS1lNm9FD9KhAAtaruv7gTdRm88QVHPf4CgA==", 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.7", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAV1lNm+VWSLBAAVrsIn004oApYgoF435tgnA==", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.0", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNmQ9y6FBAAIeW3Ce0vjSxE/AG+cOZaUQ==", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.953 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.1", 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNmdHZzHRAAXxsw24GAFGbU8daQGuE/dg==", 2024-05-27T00:40:24.954 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.2", 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNmWrk/GhAAC1qM4hIqjC5wLl1I9jdFXQ==", 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.955 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.3", 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD61VNmyPcbIxAAYh4qyXALCj9OlT/EmXla8g==", 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2024-05-27T00:40:24.956 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.admin", 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg==", 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2024-05-27T00:40:24.957 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mds", 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAg1lNmgbuuIxAARUpB9IhKnieEkeTdhvMERg==", 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mds" 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mgr", 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAg1lNmPcSuIxAAAu4wBwXH1W6inP+sOcIz/w==", 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-osd", 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAg1lNmB8yuIxAAdkiqTrc3xPcrOAFWRpWjJA==", 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-osd" 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd", 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAg1lNmv9auIxAAwptecFdhslJHIpFuYAJklQ==", 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.960 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAg1lNmJN6uIxAAo2waXt0Sh8wJnkVoDVlYjA==", 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rgw", 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQAg1lNmWeWuIxAAhi9VTZzR1I881eSfiyKn9Q==", 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.x", 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD51VNmU35lNhAA0xN2ysq6ZQy8GctphYmt+g==", 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2024-05-27T00:40:24.962 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.y", 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQD51VNmCvv9MhAA0xbdK4Zbp5x+dUa0ewT9XA==", 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-05-27T00:40:24.963 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-27T00:40:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9b7806be60 msgr2=0x7f9b7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9b7806be60 0x7f9b7806e320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c06b840 tx=0x7f9b80008040 comp rx=0 tx=0).stop 2024-05-27T00:40:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 msgr2=0x7f9b8c159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b8c159560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c011ad0 tx=0x7f9b7c011f90 comp rx=0 tx=0).stop 2024-05-27T00:40:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 shutdown_connections 2024-05-27T00:40:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9b7806be60 0x7f9b7806e320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9b8c13ae80 0x7f9b8c13b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b8c15dd90 0x7f9b8c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 --2- 172.21.15.111:0/3862330906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b74004d40 0x7f9b8c159560 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.944+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 >> 172.21.15.111:0/3862330906 conn(0x7f9b8c0b1c00 msgr2=0x7f9b8c0afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.945+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 shutdown_connections 2024-05-27T00:40:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:24.945+0000 7f9b92a58640 1 -- 172.21.15.111:0/3862330906 wait complete. 2024-05-27T00:40:24.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-27T00:40:25.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.068+0000 7fc474476640 1 Processor -- start 2024-05-27T00:40:25.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.068+0000 7fc474476640 1 -- start start 2024-05-27T00:40:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc474476640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc46c06b390 0x7fc46c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc474476640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46c0b36e0 con 0x7fc46c144000 2024-05-27T00:40:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc474476640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46c0b39c0 con 0x7fc46c0cdc80 2024-05-27T00:40:25.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc474476640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc46c0b42f0 con 0x7fc46c06b390 2024-05-27T00:40:25.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc472212640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc46c144000 0x7fc46c0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56072/0 (socket says 172.21.15.111:56072) 2024-05-27T00:40:25.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc472212640 1 -- 172.21.15.111:0/452820867 learned_addr learned my addr 172.21.15.111:0/452820867 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:25.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761794521 0 0) 0x7fc46c0b36e0 con 0x7fc46c144000 2024-05-27T00:40:25.081 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc454003520 con 0x7fc46c144000 2024-05-27T00:40:25.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.069+0000 7fc471a11640 1 --2- 172.21.15.111:0/452820867 >> v2:172.21.15.153:3300/0 conn(0x7fc46c06b390 0x7fc46c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3177051604 0 0) 0x7fc454003520 con 0x7fc46c144000 2024-05-27T00:40:25.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 >> v2:172.21.15.153:3300/0 conn(0x7fc46c06b390 msgr2=0x7fc46c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:25.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 --2- 172.21.15.111:0/452820867 >> v2:172.21.15.153:3300/0 conn(0x7fc46c06b390 0x7fc46c0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.082 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 >> v1:172.21.15.153:6789/0 conn(0x7fc46c0cdc80 legacy=0x7fc46c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c0b3ca0 con 0x7fc46c144000 2024-05-27T00:40:25.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fc45c0019f0 con 0x7fc46c144000 2024-05-27T00:40:25.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 >> v1:172.21.15.111:6789/0 conn(0x7fc46c144000 legacy=0x7fc46c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc454003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc454004800 0x7fc454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.083 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc454005210 0x7fc45400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c0b42f0 con 0x7fc454003a00 2024-05-27T00:40:25.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc45400c160 con 0x7fc454004800 2024-05-27T00:40:25.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc45400c360 con 0x7fc454005210 2024-05-27T00:40:25.084 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472212640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc454003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471a11640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc454004800 0x7fc454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472212640 1 -- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc454005210 msgr2=0x7fc45400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:25.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472a13640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc454005210 0x7fc45400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472212640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc454005210 0x7fc45400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.085 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472212640 1 -- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc454004800 msgr2=0x7fc454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:25.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472212640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc454004800 0x7fc454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472212640 1 -- 172.21.15.111:0/452820867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc454003520 con 0x7fc454003a00 2024-05-27T00:40:25.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc471a11640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc454004800 0x7fc454004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:25.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472a13640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc454005210 0x7fc45400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:25.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.070+0000 7fc472a13640 1 -- 172.21.15.111:0/452820867 reap_dead start 2024-05-27T00:40:25.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc472212640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc454003e20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00a790 tx=0x7fc454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2d22ad5dcb7f44 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00a3b0 con 0x7fc454003a00 2024-05-27T00:40:25.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c02f070 con 0x7fc454003a00 2024-05-27T00:40:25.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc471210640 1 -- 172.21.15.111:0/452820867 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c02bc40 con 0x7fc454003a00 2024-05-27T00:40:25.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 -- 172.21.15.111:0/452820867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 msgr2=0x7fc454003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.087 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc454003e20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00a790 tx=0x7fc454005830 comp rx=0 tx=0).stop 2024-05-27T00:40:25.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 -- 172.21.15.111:0/452820867 shutdown_connections 2024-05-27T00:40:25.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 --2- 172.21.15.111:0/452820867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc454003e20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 -- 172.21.15.111:0/452820867 >> 172.21.15.111:0/452820867 conn(0x7fc46c0b1c00 msgr2=0x7fc46c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:25.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 -- 172.21.15.111:0/452820867 shutdown_connections 2024-05-27T00:40:25.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.071+0000 7fc474476640 1 -- 172.21.15.111:0/452820867 wait complete. 2024-05-27T00:40:25.088 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 Processor -- start 2024-05-27T00:40:25.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 -- start start 2024-05-27T00:40:25.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc46c0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46c0a9d10 0x7fc46c0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc46c0aecf0 0x7fc46c0ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c0cd8c0 con 0x7fc454003a00 2024-05-27T00:40:25.089 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c0cd740 con 0x7fc46c0a9d10 2024-05-27T00:40:25.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc474476640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc46c0cda40 con 0x7fc46c0aecf0 2024-05-27T00:40:25.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc472212640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc46c0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc472212640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc46c0adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40316/0 (socket says 172.21.15.111:40316) 2024-05-27T00:40:25.090 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.073+0000 7fc472212640 1 -- 172.21.15.111:0/1133215909 learned_addr learned my addr 172.21.15.111:0/1133215909 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:25.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc471a11640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46c0a9d10 0x7fc46c0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472212640 1 -- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc46c0aecf0 msgr2=0x7fc46c0ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472a13640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc46c0aecf0 0x7fc46c0ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472212640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc46c0aecf0 0x7fc46c0ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472212640 1 -- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46c0a9d10 msgr2=0x7fc46c0ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472212640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46c0a9d10 0x7fc46c0ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472212640 1 -- 172.21.15.111:0/1133215909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c1f1330 con 0x7fc454003a00 2024-05-27T00:40:25.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc471a11640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46c0a9d10 0x7fc46c0ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:25.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472a13640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc46c0aecf0 0x7fc46c0ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:25.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc472212640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc46c0adf50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00a8b0 tx=0x7fc45c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c02bc70 con 0x7fc454003a00 2024-05-27T00:40:25.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45c02be10 con 0x7fc454003a00 2024-05-27T00:40:25.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c1f1610 con 0x7fc454003a00 2024-05-27T00:40:25.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0346b0 con 0x7fc454003a00 2024-05-27T00:40:25.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.074+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1f1bd0 con 0x7fc454003a00 2024-05-27T00:40:25.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.075+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46c068680 con 0x7fc454003a00 2024-05-27T00:40:25.093 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.075+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc45c047060 con 0x7fc454003a00 2024-05-27T00:40:25.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.076+0000 7fc46affd640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be60 0x7fc45806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.076+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc45c0aad80 con 0x7fc454003a00 2024-05-27T00:40:25.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.076+0000 7fc471a11640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be60 0x7fc45806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.077+0000 7fc471a11640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be60 0x7fc45806e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc46c143a10 tx=0x7fc460008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.080+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc45c048e90 con 0x7fc454003a00 2024-05-27T00:40:25.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.325+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc46c0af0d0 con 0x7fc454003a00 2024-05-27T00:40:25.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.327+0000 7fc46affd640 1 -- 172.21.15.111:0/1133215909 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fc45c0798f0 con 0x7fc454003a00 2024-05-27T00:40:25.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be60 msgr2=0x7fc45806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be60 0x7fc45806e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc46c143a10 tx=0x7fc460008040 comp rx=0 tx=0).stop 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 msgr2=0x7fc46c0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc46c0adf50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00a8b0 tx=0x7fc45c02d070 comp rx=0 tx=0).stop 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 shutdown_connections 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc45806be60 0x7fc45806e320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc46c0aecf0 0x7fc46c0ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46c0a9d10 0x7fc46c0ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 --2- 172.21.15.111:0/1133215909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc454003a00 0x7fc46c0adf50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 >> 172.21.15.111:0/1133215909 conn(0x7fc46c0b1c00 msgr2=0x7fc46c0b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 shutdown_connections 2024-05-27T00:40:25.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.329+0000 7fc474476640 1 -- 172.21.15.111:0/1133215909 wait complete. 2024-05-27T00:40:25.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-27T00:40:25.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.440+0000 7fb61c031640 1 Processor -- start 2024-05-27T00:40:25.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.440+0000 7fb61c031640 1 -- start start 2024-05-27T00:40:25.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb61c031640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb6140726d0 0x7fb614072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb61c031640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb614073020 con 0x7fb61406b390 2024-05-27T00:40:25.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb61c031640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6140b36e0 con 0x7fb614144000 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb61c031640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb6140b42f0 con 0x7fb6140726d0 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb61a5ce640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb6140726d0 0x7fb614072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb6195cc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb61406b390 0x7fb6140cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56082/0 (socket says 172.21.15.111:56082) 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.441+0000 7fb6195cc640 1 -- 172.21.15.111:0/2980431363 learned_addr learned my addr 172.21.15.111:0/2980431363 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb61a5ce640 1 -- 172.21.15.111:0/2980431363 >> v1:172.21.15.153:6789/0 conn(0x7fb614144000 legacy=0x7fb6140b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb61a5ce640 1 -- 172.21.15.111:0/2980431363 >> v1:172.21.15.111:6789/0 conn(0x7fb61406b390 legacy=0x7fb6140cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb61a5ce640 1 -- 172.21.15.111:0/2980431363 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6140b39c0 con 0x7fb6140726d0 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb61a5ce640 1 --2- 172.21.15.111:0/2980431363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6140726d0 0x7fb614072ae0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0097a0 tx=0x7fb60c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb67de4c52da5cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb618dcb640 1 -- 172.21.15.111:0/2980431363 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c037070 con 0x7fb6140726d0 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb618dcb640 1 -- 172.21.15.111:0/2980431363 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb60c032440 con 0x7fb6140726d0 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.442+0000 7fb618dcb640 1 -- 172.21.15.111:0/2980431363 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c032810 con 0x7fb6140726d0 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 -- 172.21.15.111:0/2980431363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6140726d0 msgr2=0x7fb614072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 --2- 172.21.15.111:0/2980431363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6140726d0 0x7fb614072ae0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0097a0 tx=0x7fb60c02a230 comp rx=0 tx=0).stop 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 -- 172.21.15.111:0/2980431363 shutdown_connections 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 --2- 172.21.15.111:0/2980431363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6140726d0 0x7fb614072ae0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 -- 172.21.15.111:0/2980431363 >> 172.21.15.111:0/2980431363 conn(0x7fb6140b1c00 msgr2=0x7fb6140b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:25.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 -- 172.21.15.111:0/2980431363 shutdown_connections 2024-05-27T00:40:25.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.443+0000 7fb61c031640 1 -- 172.21.15.111:0/2980431363 wait complete. 2024-05-27T00:40:25.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.444+0000 7fb61c031640 1 Processor -- start 2024-05-27T00:40:25.446 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.444+0000 7fb61c031640 1 -- start start 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61c031640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61406b390 0x7fb6140ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61c031640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6140726d0 0x7fb614152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61c031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 0x7fb6140a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61c031640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb6140cde50 con 0x7fb614144000 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61c031640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb6140b3f80 con 0x7fb6140726d0 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61c031640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb6140b4780 con 0x7fb61406b390 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 0x7fb6140a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 0x7fb6140a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40330/0 (socket says 172.21.15.111:40330) 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 -- 172.21.15.111:0/542310848 learned_addr learned my addr 172.21.15.111:0/542310848 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb6195cc640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6140726d0 0x7fb614152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb619dcd640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61406b390 0x7fb6140ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 -- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61406b390 msgr2=0x7fb6140ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61406b390 0x7fb6140ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 -- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6140726d0 msgr2=0x7fb614152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6140726d0 0x7fb614152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 -- 172.21.15.111:0/542310848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6140ac930 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.445+0000 7fb61a5ce640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 0x7fb6140a9400 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb60c035040 tx=0x7fb60c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.446+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c037040 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.446+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb60c042070 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.446+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6140a9e80 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.446+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c030b50 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.446+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61413ab70 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.447+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6140aa210 con 0x7fb614144000 2024-05-27T00:40:25.449 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.447+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb60c030cf0 con 0x7fb614144000 2024-05-27T00:40:25.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.448+0000 7fb60a7fc640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb5f806be30 0x7fb5f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.448+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fb60c0ab910 con 0x7fb614144000 2024-05-27T00:40:25.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.448+0000 7fb619dcd640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb5f806be30 0x7fb5f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.449+0000 7fb619dcd640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb5f806be30 0x7fb5f806e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb610001ca0 tx=0x7fb610009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.452+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb60c032e20 con 0x7fb614144000 2024-05-27T00:40:25.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.696+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb614068680 con 0x7fb614144000 2024-05-27T00:40:25.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.700+0000 7fb60a7fc640 1 -- 172.21.15.111:0/542310848 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fb60c07a480 con 0x7fb614144000 2024-05-27T00:40:25.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb5f806be30 msgr2=0x7fb5f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb5f806be30 0x7fb5f806e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb610001ca0 tx=0x7fb610009040 comp rx=0 tx=0).stop 2024-05-27T00:40:25.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 msgr2=0x7fb6140a9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 0x7fb6140a9400 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb60c035040 tx=0x7fb60c0046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 shutdown_connections 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb5f806be30 0x7fb5f806e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb614144000 0x7fb6140a9400 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6140726d0 0x7fb614152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 --2- 172.21.15.111:0/542310848 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb61406b390 0x7fb6140ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 >> 172.21.15.111:0/542310848 conn(0x7fb6140b1c00 msgr2=0x7fb6140af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 shutdown_connections 2024-05-27T00:40:25.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.702+0000 7fb61c031640 1 -- 172.21.15.111:0/542310848 wait complete. 2024-05-27T00:40:25.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-27T00:40:25.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.813+0000 7fc92c6d4640 1 Processor -- start 2024-05-27T00:40:25.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.813+0000 7fc92c6d4640 1 -- start start 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc92c6d4640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc9240b3630 0x7fc9240b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc92c6d4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc924162560 0x7fc9240d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc92c6d4640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc92406cc70 con 0x7fc9240b77e0 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc92c6d4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc9240d1e50 con 0x7fc924162560 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc92c6d4640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc9240d2130 con 0x7fc9240b3630 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc92ac71640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc924162560 0x7fc9240d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc929c6f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc9240b77e0 0x7fc924161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56084/0 (socket says 172.21.15.111:56084) 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.814+0000 7fc929c6f640 1 -- 172.21.15.111:0/4202689784 learned_addr learned my addr 172.21.15.111:0/4202689784 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.815+0000 7fc92ac71640 1 -- 172.21.15.111:0/4202689784 >> v2:172.21.15.153:3300/0 conn(0x7fc9240b3630 msgr2=0x7fc9240b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.815+0000 7fc92ac71640 1 --2- 172.21.15.111:0/4202689784 >> v2:172.21.15.153:3300/0 conn(0x7fc9240b3630 0x7fc9240b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.815+0000 7fc92ac71640 1 -- 172.21.15.111:0/4202689784 >> v1:172.21.15.111:6789/0 conn(0x7fc9240b77e0 legacy=0x7fc924161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.815+0000 7fc92946e640 1 -- 172.21.15.111:0/4202689784 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305206344 0 0) 0x7fc92406cc70 con 0x7fc9240b77e0 2024-05-27T00:40:25.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.815+0000 7fc92ac71640 1 -- 172.21.15.111:0/4202689784 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240d2410 con 0x7fc924162560 2024-05-27T00:40:25.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.815+0000 7fc92ac71640 1 --2- 172.21.15.111:0/4202689784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc924162560 0x7fc9240d2c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc92408ccb0 tx=0x7fc91c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59104fa217784cc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92946e640 1 -- 172.21.15.111:0/4202689784 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c037070 con 0x7fc924162560 2024-05-27T00:40:25.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92946e640 1 -- 172.21.15.111:0/4202689784 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c032440 con 0x7fc924162560 2024-05-27T00:40:25.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92946e640 1 -- 172.21.15.111:0/4202689784 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c032810 con 0x7fc924162560 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 -- 172.21.15.111:0/4202689784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc924162560 msgr2=0x7fc9240d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/4202689784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc924162560 0x7fc9240d2c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc92408ccb0 tx=0x7fc91c02a1a0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 -- 172.21.15.111:0/4202689784 shutdown_connections 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/4202689784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc924162560 0x7fc9240d2c10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/4202689784 >> v2:172.21.15.153:3300/0 conn(0x7fc9240b3630 0x7fc9240b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 -- 172.21.15.111:0/4202689784 >> 172.21.15.111:0/4202689784 conn(0x7fc9240d42c0 msgr2=0x7fc9240d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 -- 172.21.15.111:0/4202689784 shutdown_connections 2024-05-27T00:40:25.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.816+0000 7fc92c6d4640 1 -- 172.21.15.111:0/4202689784 wait complete. 2024-05-27T00:40:25.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.817+0000 7fc92c6d4640 1 Processor -- start 2024-05-27T00:40:25.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.817+0000 7fc92c6d4640 1 -- start start 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc92c6d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9240b3630 0x7fc924157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc92c6d4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 0x7fc9241e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc92c6d4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc924162560 0x7fc9241e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc92c6d4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240d32d0 con 0x7fc9240b77e0 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc92c6d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240d3150 con 0x7fc9240b3630 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc92c6d4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc9240d3450 con 0x7fc924162560 2024-05-27T00:40:25.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc929c6f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 0x7fc9241e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc929c6f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 0x7fc9241e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40358/0 (socket says 172.21.15.111:40358) 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc929c6f640 1 -- 172.21.15.111:0/3899159746 learned_addr learned my addr 172.21.15.111:0/3899159746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc929c6f640 1 -- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc924162560 msgr2=0x7fc9241e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc929c6f640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc924162560 0x7fc9241e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.818+0000 7fc929c6f640 1 -- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9240b3630 msgr2=0x7fc924157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc929c6f640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9240b3630 0x7fc924157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc929c6f640 1 -- 172.21.15.111:0/3899159746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240c7d20 con 0x7fc9240b77e0 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc929c6f640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 0x7fc9241e83a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c4b0 tx=0x7fc91800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918018070 con 0x7fc9240b77e0 2024-05-27T00:40:25.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc918007500 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9240c8000 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918004a20 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.819+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9240c85c0 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.820+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc924068680 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.820+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc91801b070 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.821+0000 7fc922ffd640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc91006be30 0x7fc91006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.821+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc918089f40 con 0x7fc9240b77e0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.821+0000 7fc92a470640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc91006be30 0x7fc91006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:25.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.822+0000 7fc92a470640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc91006be30 0x7fc91006e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc91400ab90 tx=0x7fc914009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:25.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:25.825+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc918058c00 con 0x7fc9240b77e0 2024-05-27T00:40:26.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.064+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc9240c89e0 con 0x7fc9240b77e0 2024-05-27T00:40:26.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.068+0000 7fc922ffd640 1 -- 172.21.15.111:0/3899159746 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fc918058500 con 0x7fc9240b77e0 2024-05-27T00:40:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc91006be30 msgr2=0x7fc91006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc91006be30 0x7fc91006e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc91400ab90 tx=0x7fc914009250 comp rx=0 tx=0).stop 2024-05-27T00:40:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 msgr2=0x7fc9241e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 0x7fc9241e83a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c4b0 tx=0x7fc91800c970 comp rx=0 tx=0).stop 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 shutdown_connections 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc91006be30 0x7fc91006e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc924162560 0x7fc9241e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9240b77e0 0x7fc9241e83a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 --2- 172.21.15.111:0/3899159746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9240b3630 0x7fc924157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 >> 172.21.15.111:0/3899159746 conn(0x7fc9240d42c0 msgr2=0x7fc9240d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 shutdown_connections 2024-05-27T00:40:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.070+0000 7fc92c6d4640 1 -- 172.21.15.111:0/3899159746 wait complete. 2024-05-27T00:40:26.082 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-27T00:40:26.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.181+0000 7f1f6e472640 1 Processor -- start 2024-05-27T00:40:26.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.181+0000 7f1f6e472640 1 -- start start 2024-05-27T00:40:26.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6e472640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f6813ef60 0x7f1f680cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6e472640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813f340 0x7f1f680cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6e472640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f68142650 con 0x7f1f680cce00 2024-05-27T00:40:26.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6e472640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680cf210 con 0x7f1f6813ef60 2024-05-27T00:40:26.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6e472640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1f680cebc0 con 0x7f1f6813f340 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f67fff640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f6813ef60 0x7f1f680cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6ca0f640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1f680cce00 0x7f1f68141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56094/0 (socket says 172.21.15.111:56094) 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.182+0000 7f1f6ca0f640 1 -- 172.21.15.111:0/3486189394 learned_addr learned my addr 172.21.15.111:0/3486189394 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f5ffff640 1 --2- 172.21.15.111:0/3486189394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813f340 0x7f1f680cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f67fff640 1 -- 172.21.15.111:0/3486189394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813f340 msgr2=0x7f1f680cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f67fff640 1 --2- 172.21.15.111:0/3486189394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813f340 0x7f1f680cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f67fff640 1 -- 172.21.15.111:0/3486189394 >> v1:172.21.15.111:6789/0 conn(0x7f1f680cce00 legacy=0x7f1f68141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f67fff640 1 -- 172.21.15.111:0/3486189394 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f680ceea0 con 0x7f1f6813ef60 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f677fe640 1 -- 172.21.15.111:0/3486189394 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907518239 0 0) 0x7f1f68142650 con 0x7f1f680cce00 2024-05-27T00:40:26.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.183+0000 7f1f67fff640 1 --2- 172.21.15.111:0/3486189394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f6813ef60 0x7f1f680cf6a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f580097a0 tx=0x7f1f5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cc140aa3402ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f677fe640 1 -- 172.21.15.111:0/3486189394 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58037070 con 0x7f1f6813ef60 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f677fe640 1 -- 172.21.15.111:0/3486189394 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f58032440 con 0x7f1f6813ef60 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f677fe640 1 -- 172.21.15.111:0/3486189394 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58032810 con 0x7f1f6813ef60 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 -- 172.21.15.111:0/3486189394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f6813ef60 msgr2=0x7f1f680cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 --2- 172.21.15.111:0/3486189394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f6813ef60 0x7f1f680cf6a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f580097a0 tx=0x7f1f5802a180 comp rx=0 tx=0).stop 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 -- 172.21.15.111:0/3486189394 shutdown_connections 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 --2- 172.21.15.111:0/3486189394 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813f340 0x7f1f680cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 --2- 172.21.15.111:0/3486189394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f6813ef60 0x7f1f680cf6a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 -- 172.21.15.111:0/3486189394 >> 172.21.15.111:0/3486189394 conn(0x7f1f680b1c00 msgr2=0x7f1f680b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 -- 172.21.15.111:0/3486189394 shutdown_connections 2024-05-27T00:40:26.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.184+0000 7f1f6e472640 1 -- 172.21.15.111:0/3486189394 wait complete. 2024-05-27T00:40:26.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.185+0000 7f1f6e472640 1 Processor -- start 2024-05-27T00:40:26.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.185+0000 7f1f6e472640 1 -- start start 2024-05-27T00:40:26.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f6e472640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 0x7f1f6813e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f6e472640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813ef60 0x7f1f6813f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f6e472640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f6813f340 0x7f1f681411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f6e472640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68142030 con 0x7f1f680cce00 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f6e472640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1f680ce890 con 0x7f1f6813f340 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f6e472640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1f681421b0 con 0x7f1f6813ef60 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 0x7f1f6813e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 0x7f1f6813e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40380/0 (socket says 172.21.15.111:40380) 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.186+0000 7f1f67fff640 1 -- 172.21.15.111:0/2015805602 learned_addr learned my addr 172.21.15.111:0/2015805602 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f67fff640 1 -- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813ef60 msgr2=0x7f1f6813f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f67fff640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813ef60 0x7f1f6813f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f67fff640 1 -- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f6813f340 msgr2=0x7f1f681411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f67fff640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f6813f340 0x7f1f681411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f67fff640 1 -- 172.21.15.111:0/2015805602 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f68142eb0 con 0x7f1f680cce00 2024-05-27T00:40:26.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f67fff640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 0x7f1f6813e8b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f58035040 tx=0x7f1f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58037050 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f58038040 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58004470 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6813fe90 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.187+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f68140450 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.188+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f68068680 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.188+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f58030900 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.189+0000 7f1f657fa640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f4806be60 0x7f1f4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.189+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1f580ab6c0 con 0x7f1f680cce00 2024-05-27T00:40:26.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.189+0000 7f1f5ffff640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f4806be60 0x7f1f4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.190+0000 7f1f5ffff640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f4806be60 0x7f1f4806e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006fd0 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.194 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.193+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1f5807a2c0 con 0x7f1f680cce00 2024-05-27T00:40:26.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.433+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1f68160860 con 0x7f1f680cce00 2024-05-27T00:40:26.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.437+0000 7f1f657fa640 1 -- 172.21.15.111:0/2015805602 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f1f58079bc0 con 0x7f1f680cce00 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f4806be60 msgr2=0x7f1f4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f4806be60 0x7f1f4806e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006fd0 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 msgr2=0x7f1f6813e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 0x7f1f6813e8b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f58035040 tx=0x7f1f580046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 shutdown_connections 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1f4806be60 0x7f1f4806e320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f6813f340 0x7f1f681411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.441 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1f6813ef60 0x7f1f6813f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 --2- 172.21.15.111:0/2015805602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f680cce00 0x7f1f6813e8b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 >> 172.21.15.111:0/2015805602 conn(0x7f1f680b1c00 msgr2=0x7f1f680afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 shutdown_connections 2024-05-27T00:40:26.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.440+0000 7f1f6e472640 1 -- 172.21.15.111:0/2015805602 wait complete. 2024-05-27T00:40:26.452 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-27T00:40:26.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.551+0000 7f1ab7a09640 1 Processor -- start 2024-05-27T00:40:26.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.551+0000 7f1ab7a09640 1 -- start start 2024-05-27T00:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab7a09640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 0x7f1ab00cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab7a09640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab00b36e0 con 0x7f1ab0144000 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab7a09640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab00b39c0 con 0x7f1ab00cdc80 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab7a09640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab00b42f0 con 0x7f1ab006b390 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab57a5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1ab0144000 0x7f1ab00b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56098/0 (socket says 172.21.15.111:56098) 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab57a5640 1 -- 172.21.15.111:0/3116798083 learned_addr learned my addr 172.21.15.111:0/3116798083 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178773660 0 0) 0x7f1ab00b36e0 con 0x7f1ab0144000 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a9c003520 con 0x7f1ab0144000 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.552+0000 7f1ab4fa4640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 0x7f1ab00cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2655333473 0 0) 0x7f1a9c003520 con 0x7f1ab0144000 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 msgr2=0x7f1ab00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 0x7f1ab00cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 >> v1:172.21.15.153:6789/0 conn(0x7f1ab00cdc80 legacy=0x7f1ab00726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab00b3ca0 con 0x7f1ab0144000 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f1aac001670 con 0x7f1ab0144000 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 >> v1:172.21.15.111:6789/0 conn(0x7f1ab0144000 legacy=0x7f1ab00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1a9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 0x7f1a9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00b42f0 con 0x7f1a9c0039e0 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c00baa0 con 0x7f1a9c004d40 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9c00bca0 con 0x7f1ab006b390 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1a9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab4fa4640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 0x7f1a9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab5fa6640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab57a5640 1 -- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 msgr2=0x7f1a9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab006b390 0x7f1a9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab57a5640 1 -- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a9c004d40 msgr2=0x7f1a9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab57a5640 1 -- 172.21.15.111:0/3116798083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c003520 con 0x7f1a9c0039e0 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.553+0000 7f1ab5fa6640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a9c004d40 0x7f1a9c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1a9c003e00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001050 tx=0x7f1a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee673c6dbdebb0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac007d10 con 0x7f1a9c0039e0 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac005ad0 con 0x7f1a9c0039e0 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1aa7fff640 1 -- 172.21.15.111:0/3116798083 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac007070 con 0x7f1a9c0039e0 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab7a09640 1 -- 172.21.15.111:0/3116798083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 msgr2=0x7f1a9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/3116798083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1a9c003e00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001050 tx=0x7f1a9c005830 comp rx=0 tx=0).stop 2024-05-27T00:40:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab5fa6640 1 -- 172.21.15.111:0/3116798083 reap_dead start 2024-05-27T00:40:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab7a09640 1 -- 172.21.15.111:0/3116798083 shutdown_connections 2024-05-27T00:40:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab7a09640 1 -- 172.21.15.111:0/3116798083 >> 172.21.15.111:0/3116798083 conn(0x7f1ab00b1c00 msgr2=0x7f1ab00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab7a09640 1 -- 172.21.15.111:0/3116798083 shutdown_connections 2024-05-27T00:40:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.554+0000 7f1ab7a09640 1 -- 172.21.15.111:0/3116798083 wait complete. 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 Processor -- start 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 -- start start 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1ab0133040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab0134b10 0x7f1ab00cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab00cba20 0x7f1ab01336b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0072f60 con 0x7f1a9c0039e0 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0072de0 con 0x7f1ab0134b10 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab7a09640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab00cd740 con 0x7f1ab00cba20 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab57a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1ab0133040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab57a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1ab0133040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40390/0 (socket says 172.21.15.111:40390) 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.556+0000 7f1ab57a5640 1 -- 172.21.15.111:0/2455867464 learned_addr learned my addr 172.21.15.111:0/2455867464 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab5fa6640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab00cba20 0x7f1ab01336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab4fa4640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab0134b10 0x7f1ab00cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab57a5640 1 -- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab00cba20 msgr2=0x7f1ab01336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab00cba20 0x7f1ab01336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab57a5640 1 -- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab0134b10 msgr2=0x7f1ab00cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab0134b10 0x7f1ab00cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab57a5640 1 -- 172.21.15.111:0/2455867464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab0134f90 con 0x7f1a9c0039e0 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab57a5640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1ab0133040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c300 tx=0x7f1ab006f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac001630 con 0x7f1a9c0039e0 2024-05-27T00:40:26.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac042070 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab0135240 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac007270 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.557+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab0135800 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.558+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab0068680 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.559+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1aac007410 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.559+0000 7f1aa5ffb640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1a9406be60 0x7f1a9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.559+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1aac0aad00 con 0x7f1a9c0039e0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.560+0000 7f1ab4fa4640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1a9406be60 0x7f1a9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.560+0000 7f1ab4fa4640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1a9406be60 0x7f1a9406e320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1ab01407e0 tx=0x7f1aa0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.564+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1aac079900 con 0x7f1a9c0039e0 2024-05-27T00:40:26.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.811+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f1ab0135ec0 con 0x7f1a9c0039e0 2024-05-27T00:40:26.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.813+0000 7f1aa5ffb640 1 -- 172.21.15.111:0/2455867464 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) ==== 97+0+0 (secure 0 0 0) 0x7f1aac005aa0 con 0x7f1a9c0039e0 2024-05-27T00:40:26.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1a9406be60 msgr2=0x7f1a9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1a9406be60 0x7f1a9406e320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1ab01407e0 tx=0x7f1aa0008040 comp rx=0 tx=0).stop 2024-05-27T00:40:26.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 msgr2=0x7f1ab0133040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1ab0133040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c300 tx=0x7f1ab006f710 comp rx=0 tx=0).stop 2024-05-27T00:40:26.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 shutdown_connections 2024-05-27T00:40:26.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1a9406be60 0x7f1a9406e320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1ab00cba20 0x7f1ab01336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ab0134b10 0x7f1ab00cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 --2- 172.21.15.111:0/2455867464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a9c0039e0 0x7f1ab0133040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 >> 172.21.15.111:0/2455867464 conn(0x7f1ab00b1c00 msgr2=0x7f1ab0071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:26.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 shutdown_connections 2024-05-27T00:40:26.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.816+0000 7f1ab7a09640 1 -- 172.21.15.111:0/2455867464 wait complete. 2024-05-27T00:40:26.831 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-27T00:40:26.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.931+0000 7f3d15688640 1 Processor -- start 2024-05-27T00:40:26.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.931+0000 7f3d15688640 1 -- start start 2024-05-27T00:40:26.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.931+0000 7f3d15688640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d100b3630 0x7f3d100b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.931+0000 7f3d15688640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1006cc70 con 0x7f3d100b77e0 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.931+0000 7f3d15688640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d100d1e50 con 0x7f3d10162560 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.931+0000 7f3d15688640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f3d100d2130 con 0x7f3d100b3630 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0e7fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3d100b77e0 0x7f3d10161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56110/0 (socket says 172.21.15.111:56110) 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0e7fc640 1 -- 172.21.15.111:0/555720036 learned_addr learned my addr 172.21.15.111:0/555720036 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0effd640 1 --2- 172.21.15.111:0/555720036 >> v2:172.21.15.153:3300/0 conn(0x7f3d100b3630 0x7f3d100b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426588566 0 0) 0x7f3d1006cc70 con 0x7f3d100b77e0 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cf8003520 con 0x7f3d100b77e0 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 408149560 0 0) 0x7f3cf8003520 con 0x7f3d100b77e0 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 >> v2:172.21.15.153:3300/0 conn(0x7f3d100b3630 msgr2=0x7f3d100b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 --2- 172.21.15.111:0/555720036 >> v2:172.21.15.153:3300/0 conn(0x7f3d100b3630 0x7f3d100b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 >> v1:172.21.15.153:6789/0 conn(0x7f3d10162560 legacy=0x7f3d100d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d100d2410 con 0x7f3d100b77e0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f3d04001670 con 0x7f3d100b77e0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 >> v1:172.21.15.111:6789/0 conn(0x7f3d100b77e0 legacy=0x7f3d10161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80039b0 0x7f3cf8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf8004800 0x7f3cf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf80051c0 0x7f3cf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d100d2130 con 0x7f3cf80051c0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800c060 con 0x7f3cf8004800 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf800c260 con 0x7f3cf80039b0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf80051c0 0x7f3cf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0e7fc640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf8004800 0x7f3cf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.932+0000 7f3d0effd640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80039b0 0x7f3cf8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80039b0 msgr2=0x7f3cf8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80039b0 0x7f3cf8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf8004800 msgr2=0x7f3cf8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cf8004800 0x7f3cf8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/555720036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf8003520 con 0x7f3cf80051c0 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/555720036 reap_dead start 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf80051c0 0x7f3cf800b9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d1008d4a0 tx=0x7f3cf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2809ab5285abfbb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08001370 con 0x7f3cf80051c0 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d080052d0 con 0x7f3cf80051c0 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.933+0000 7f3d0dffb640 1 -- 172.21.15.111:0/555720036 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0802ab30 con 0x7f3cf80051c0 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 -- 172.21.15.111:0/555720036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf80051c0 msgr2=0x7f3cf800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf80051c0 0x7f3cf800b9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d1008d4a0 tx=0x7f3cf8005830 comp rx=0 tx=0).stop 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 -- 172.21.15.111:0/555720036 shutdown_connections 2024-05-27T00:40:26.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 --2- 172.21.15.111:0/555720036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf80051c0 0x7f3cf800b9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 -- 172.21.15.111:0/555720036 >> 172.21.15.111:0/555720036 conn(0x7f3d100d42c0 msgr2=0x7f3d100d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:26.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 -- 172.21.15.111:0/555720036 shutdown_connections 2024-05-27T00:40:26.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.934+0000 7f3d15688640 1 -- 172.21.15.111:0/555720036 wait complete. 2024-05-27T00:40:26.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.935+0000 7f3d15688640 1 Processor -- start 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.935+0000 7f3d15688640 1 -- start start 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d15688640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80051c0 0x7f3d10159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d15688640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d100b4b20 0x7f3d100b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d15688640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 0x7f3d100c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d15688640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d100b7420 con 0x7f3d100c9320 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d15688640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f3d100b72a0 con 0x7f3d100b4b20 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d15688640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f3d100b75a0 con 0x7f3cf80051c0 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 0x7f3d100c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 0x7f3d100c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40396/0 (socket says 172.21.15.111:40396) 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/1256706362 learned_addr learned my addr 172.21.15.111:0/1256706362 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:26.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0effd640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80051c0 0x7f3d10159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0e7fc640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d100b4b20 0x7f3d100b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80051c0 msgr2=0x7f3d10159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80051c0 0x7f3d10159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d100b4b20 msgr2=0x7f3d100b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d100b4b20 0x7f3d100b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0f7fe640 1 -- 172.21.15.111:0/1256706362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d10159e80 con 0x7f3d100c9320 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.936+0000 7f3d0e7fc640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d100b4b20 0x7f3d100b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.937+0000 7f3d0f7fe640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 0x7f3d100c97b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d0802a500 tx=0x7f3d08001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.937+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08007a30 con 0x7f3d100c9320 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.937+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d08005480 con 0x7f3d100c9320 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.937+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d100ca010 con 0x7f3d100c9320 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.937+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08004b20 con 0x7f3d100c9320 2024-05-27T00:40:26.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.937+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d100ca5d0 con 0x7f3d100c9320 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.938+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d100b54e0 con 0x7f3d100c9320 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.938+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d08004cc0 con 0x7f3d100c9320 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.939+0000 7f3cf77fe640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3cfc06be30 0x7f3cfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.939+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f3d080a9f90 con 0x7f3d100c9320 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.939+0000 7f3d0effd640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3cfc06be30 0x7f3cfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.939+0000 7f3d0effd640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3cfc06be30 0x7f3cfc06e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ce340 tx=0x7f3d0000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:26.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:26.943+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d08078b90 con 0x7f3d100c9320 2024-05-27T00:40:27.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.186+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f3d100cef10 con 0x7f3d100c9320 2024-05-27T00:40:27.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.188+0000 7f3cf77fe640 1 -- 172.21.15.111:0/1256706362 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f3d08078490 con 0x7f3d100c9320 2024-05-27T00:40:27.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3cfc06be30 msgr2=0x7f3cfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3cfc06be30 0x7f3cfc06e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ce340 tx=0x7f3d0000b040 comp rx=0 tx=0).stop 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 msgr2=0x7f3d100c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 0x7f3d100c97b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d0802a500 tx=0x7f3d08001120 comp rx=0 tx=0).stop 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 shutdown_connections 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f3cfc06be30 0x7f3cfc06e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d100c9320 0x7f3d100c97b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d100b4b20 0x7f3d100b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 --2- 172.21.15.111:0/1256706362 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f3cf80051c0 0x7f3d10159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 >> 172.21.15.111:0/1256706362 conn(0x7f3d100d42c0 msgr2=0x7f3d10161690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 shutdown_connections 2024-05-27T00:40:27.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.191+0000 7f3d15688640 1 -- 172.21.15.111:0/1256706362 wait complete. 2024-05-27T00:40:27.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:27.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.303+0000 7f094b829640 1 Processor -- start 2024-05-27T00:40:27.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 -- start start 2024-05-27T00:40:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f09440b3630 0x7f09440b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f09440b77e0 0x7f0944161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0944162390 0x7f09440d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f094406cc70 con 0x7f09440b77e0 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f09440d1e50 con 0x7f09440b3630 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f094b829640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f09440d2130 con 0x7f0944162390 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.304+0000 7f0948dc4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f09440b77e0 0x7f0944161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f09440b77e0 0x7f0944161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40408/0 (socket says 172.21.15.111:40408) 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 -- 172.21.15.111:0/1289754443 learned_addr learned my addr 172.21.15.111:0/1289754443 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 -- 172.21.15.111:0/1289754443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0944162390 msgr2=0x7f09440d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 --2- 172.21.15.111:0/1289754443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0944162390 0x7f09440d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 -- 172.21.15.111:0/1289754443 >> v2:172.21.15.153:3300/0 conn(0x7f09440b3630 msgr2=0x7f09440b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f09495c5640 1 --2- 172.21.15.111:0/1289754443 >> v2:172.21.15.153:3300/0 conn(0x7f09440b3630 0x7f09440b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 --2- 172.21.15.111:0/1289754443 >> v2:172.21.15.153:3300/0 conn(0x7f09440b3630 0x7f09440b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f0948dc4640 1 -- 172.21.15.111:0/1289754443 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09440d2410 con 0x7f09440b77e0 2024-05-27T00:40:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.305+0000 7f09495c5640 1 --2- 172.21.15.111:0/1289754443 >> v2:172.21.15.153:3300/0 conn(0x7f09440b3630 0x7f09440b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f0948dc4640 1 --2- 172.21.15.111:0/1289754443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09440b77e0 0x7f0944161e50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f094408ccb0 tx=0x7f093402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb351878e909598 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f093bfff640 1 -- 172.21.15.111:0/1289754443 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934037070 con 0x7f09440b77e0 2024-05-27T00:40:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f093bfff640 1 -- 172.21.15.111:0/1289754443 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0934032440 con 0x7f09440b77e0 2024-05-27T00:40:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f093bfff640 1 -- 172.21.15.111:0/1289754443 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934032810 con 0x7f09440b77e0 2024-05-27T00:40:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 -- 172.21.15.111:0/1289754443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09440b77e0 msgr2=0x7f0944161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 --2- 172.21.15.111:0/1289754443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09440b77e0 0x7f0944161e50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f094408ccb0 tx=0x7f093402a1e0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 -- 172.21.15.111:0/1289754443 shutdown_connections 2024-05-27T00:40:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 --2- 172.21.15.111:0/1289754443 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0944162390 0x7f09440d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 --2- 172.21.15.111:0/1289754443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09440b77e0 0x7f0944161e50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 --2- 172.21.15.111:0/1289754443 >> v2:172.21.15.153:3300/0 conn(0x7f09440b3630 0x7f09440b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 -- 172.21.15.111:0/1289754443 >> 172.21.15.111:0/1289754443 conn(0x7f09440d42c0 msgr2=0x7f09440d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 -- 172.21.15.111:0/1289754443 shutdown_connections 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.306+0000 7f094b829640 1 -- 172.21.15.111:0/1289754443 wait complete. 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.307+0000 7f094b829640 1 Processor -- start 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 -- start start 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09440b3630 0x7f09441e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09440b77e0 0x7f09440ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 0x7f09441602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f09440d32d0 con 0x7f0944162390 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f09440d3150 con 0x7f09440b3630 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f094b829640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f09440d3450 con 0x7f09440b77e0 2024-05-27T00:40:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f0949dc6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 0x7f09441602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f0949dc6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 0x7f09441602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40412/0 (socket says 172.21.15.111:40412) 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.308+0000 7f0949dc6640 1 -- 172.21.15.111:0/3882243306 learned_addr learned my addr 172.21.15.111:0/3882243306 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0948dc4640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09440b77e0 0x7f09440ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f09495c5640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09440b3630 0x7f09441e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0949dc6640 1 -- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09440b77e0 msgr2=0x7f09440ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0949dc6640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09440b77e0 0x7f09440ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0949dc6640 1 -- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09440b3630 msgr2=0x7f09441e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0949dc6640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09440b3630 0x7f09441e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0949dc6640 1 -- 172.21.15.111:0/3882243306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09440cb350 con 0x7f0944162390 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0948dc4640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09440b77e0 0x7f09440ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f09495c5640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09440b3630 0x7f09441e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0949dc6640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 0x7f09441602e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f093c009fd0 tx=0x7f093c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c011830 con 0x7f0944162390 2024-05-27T00:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c016070 con 0x7f0944162390 2024-05-27T00:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c011bf0 con 0x7f0944162390 2024-05-27T00:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09440cb630 con 0x7f0944162390 2024-05-27T00:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.309+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09440cbbf0 con 0x7f0944162390 2024-05-27T00:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.310+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0944161e50 con 0x7f0944162390 2024-05-27T00:40:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.311+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f093c01b460 con 0x7f0944162390 2024-05-27T00:40:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.311+0000 7f0939ffb640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f092806be30 0x7f092806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.311+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f093c08aa30 con 0x7f0944162390 2024-05-27T00:40:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.311+0000 7f09495c5640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f092806be30 0x7f092806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.312+0000 7f09495c5640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f092806be30 0x7f092806e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0940007f40 tx=0x7f0940007490 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.315+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0944161e50 con 0x7f0944162390 2024-05-27T00:40:27.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.491+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09440cef10 con 0x7f0944162390 2024-05-27T00:40:27.492 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.491+0000 7f0939ffb640 1 -- 172.21.15.111:0/3882243306 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13212 (secure 0 0 0) 0x7f093c0595a0 con 0x7f0944162390 2024-05-27T00:40:27.492 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:27.493 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":35,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:24.577011+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f092806be30 msgr2=0x7f092806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f092806be30 0x7f092806e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0940007f40 tx=0x7f0940007490 comp rx=0 tx=0).stop 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 msgr2=0x7f09441602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 0x7f09441602e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f093c009fd0 tx=0x7f093c00b8e0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 shutdown_connections 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f092806be30 0x7f092806e2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0944162390 0x7f09441602e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f09440b77e0 0x7f09440ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.493+0000 7f094b829640 1 --2- 172.21.15.111:0/3882243306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09440b3630 0x7f09441e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.495 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.494+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 >> 172.21.15.111:0/3882243306 conn(0x7f09440d42c0 msgr2=0x7f09440b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:27.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.494+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 shutdown_connections 2024-05-27T00:40:27.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.494+0000 7f094b829640 1 -- 172.21.15.111:0/3882243306 wait complete. 2024-05-27T00:40:27.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T00:40:27.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T00:40:27.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.609+0000 7f03df2a5640 1 Processor -- start 2024-05-27T00:40:27.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.609+0000 7f03df2a5640 1 -- start start 2024-05-27T00:40:27.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03df2a5640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f03d806b390 0x7f03d80cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03df2a5640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d80b36e0 con 0x7f03d8144000 2024-05-27T00:40:27.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03df2a5640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d80b39c0 con 0x7f03d80cdc80 2024-05-27T00:40:27.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03df2a5640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f03d80b42f0 con 0x7f03d806b390 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03dc840640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f03d806b390 0x7f03d80cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03dd041640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f03d8144000 0x7f03d80b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:56126/0 (socket says 172.21.15.111:56126) 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03dd041640 1 -- 172.21.15.111:0/3047015119 learned_addr learned my addr 172.21.15.111:0/3047015119 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03dc840640 1 -- 172.21.15.111:0/3047015119 >> v1:172.21.15.153:6789/0 conn(0x7f03d80cdc80 legacy=0x7f03d80726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03dc840640 1 -- 172.21.15.111:0/3047015119 >> v1:172.21.15.111:6789/0 conn(0x7f03d8144000 legacy=0x7f03d80b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03dc840640 1 -- 172.21.15.111:0/3047015119 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d80b3ca0 con 0x7f03d806b390 2024-05-27T00:40:27.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.610+0000 7f03cffff640 1 -- 172.21.15.111:0/3047015119 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 77155981 0 0) 0x7f03d80b36e0 con 0x7f03d8144000 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.611+0000 7f03dc840640 1 --2- 172.21.15.111:0/3047015119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d806b390 0x7f03d80cd740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a601022c808fd03 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.611+0000 7f03cffff640 1 -- 172.21.15.111:0/3047015119 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8037070 con 0x7f03d806b390 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.611+0000 7f03cffff640 1 -- 172.21.15.111:0/3047015119 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c8032440 con 0x7f03d806b390 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.611+0000 7f03cffff640 1 -- 172.21.15.111:0/3047015119 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8032810 con 0x7f03d806b390 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 -- 172.21.15.111:0/3047015119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d806b390 msgr2=0x7f03d80cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 --2- 172.21.15.111:0/3047015119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d806b390 0x7f03d80cd740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c802a1e0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 -- 172.21.15.111:0/3047015119 shutdown_connections 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 --2- 172.21.15.111:0/3047015119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d806b390 0x7f03d80cd740 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 -- 172.21.15.111:0/3047015119 >> 172.21.15.111:0/3047015119 conn(0x7f03d80b1c00 msgr2=0x7f03d80b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 -- 172.21.15.111:0/3047015119 shutdown_connections 2024-05-27T00:40:27.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.612+0000 7f03df2a5640 1 -- 172.21.15.111:0/3047015119 wait complete. 2024-05-27T00:40:27.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.613+0000 7f03df2a5640 1 Processor -- start 2024-05-27T00:40:27.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.613+0000 7f03df2a5640 1 -- start start 2024-05-27T00:40:27.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03df2a5640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03d806b390 0x7f03d815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03df2a5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 0x7f03d813a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03df2a5640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03d8144000 0x7f03d815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03df2a5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8072f60 con 0x7f03d80cdc80 2024-05-27T00:40:27.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03df2a5640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8072de0 con 0x7f03d8144000 2024-05-27T00:40:27.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03df2a5640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f03d80b4780 con 0x7f03d806b390 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 0x7f03d813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 0x7f03d813a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40428/0 (socket says 172.21.15.111:40428) 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 -- 172.21.15.111:0/2558728020 learned_addr learned my addr 172.21.15.111:0/2558728020 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dd842640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03d8144000 0x7f03d815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dd041640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03d806b390 0x7f03d815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 -- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03d806b390 msgr2=0x7f03d815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03d806b390 0x7f03d815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 -- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03d8144000 msgr2=0x7f03d815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03d8144000 0x7f03d815bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 -- 172.21.15.111:0/2558728020 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d8157870 con 0x7f03d80cdc80 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03dc840640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 0x7f03d813a990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.614+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8041070 con 0x7f03d80cdc80 2024-05-27T00:40:27.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.615+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c803c870 con 0x7f03d80cdc80 2024-05-27T00:40:27.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.615+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8037040 con 0x7f03d80cdc80 2024-05-27T00:40:27.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.615+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d8157b50 con 0x7f03d80cdc80 2024-05-27T00:40:27.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.615+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d8158110 con 0x7f03d80cdc80 2024-05-27T00:40:27.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.616+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d8068680 con 0x7f03d80cdc80 2024-05-27T00:40:27.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.616+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03c8044070 con 0x7f03d80cdc80 2024-05-27T00:40:27.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.617+0000 7f03cdffb640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f03bc06be60 0x7f03bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.617+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f03c80abac0 con 0x7f03d80cdc80 2024-05-27T00:40:27.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.617+0000 7f03dd041640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f03bc06be60 0x7f03bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.620 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.618+0000 7f03dd041640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f03bc06be60 0x7f03bc06e320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000f60 tx=0x7f03d4007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.621 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.621+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03c807a7e0 con 0x7f03d80cdc80 2024-05-27T00:40:27.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.793+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03d815c4b0 con 0x7f03d80cdc80 2024-05-27T00:40:27.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.794+0000 7f03cdffb640 1 -- 172.21.15.111:0/2558728020 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f03c807a0e0 con 0x7f03d80cdc80 2024-05-27T00:40:27.795 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T00:40:27.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.795+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f03bc06be60 msgr2=0x7f03bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.795+0000 7f03df2a5640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f03bc06be60 0x7f03bc06e320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000f60 tx=0x7f03d4007c00 comp rx=0 tx=0).stop 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.795+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 msgr2=0x7f03d813a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.795+0000 7f03df2a5640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 0x7f03d813a990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c80046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 shutdown_connections 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f03bc06be60 0x7f03bc06e320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03d8144000 0x7f03d815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03d80cdc80 0x7f03d813a990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 --2- 172.21.15.111:0/2558728020 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f03d806b390 0x7f03d815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 >> 172.21.15.111:0/2558728020 conn(0x7f03d80b1c00 msgr2=0x7f03d8141f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 shutdown_connections 2024-05-27T00:40:27.797 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.796+0000 7f03df2a5640 1 -- 172.21.15.111:0/2558728020 wait complete. 2024-05-27T00:40:27.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-27T00:40:27.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.905+0000 7fa416488640 1 Processor -- start 2024-05-27T00:40:27.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.905+0000 7fa416488640 1 -- start start 2024-05-27T00:40:27.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa416488640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4100cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa416488640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 0x7fa4100726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa416488640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4100b36e0 con 0x7fa410144000 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa416488640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa4100b42f0 con 0x7fa4100cdc80 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa416488640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa4100b39c0 con 0x7fa41006b390 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa414a25640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 0x7fa4100726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa40f7fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4100cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa414a25640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 0x7fa4100726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:39374/0 (socket says 172.21.15.111:39374) 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa414a25640 1 -- 172.21.15.111:0/2263436685 learned_addr learned my addr 172.21.15.111:0/2263436685 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa40f7fe640 1 --2- 172.21.15.111:0/2263436685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4100cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:39370/0 (socket says 172.21.15.111:39370) 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.906+0000 7fa40ffff640 1 --1- 172.21.15.111:0/2263436685 >> v1:172.21.15.153:6789/0 conn(0x7fa410144000 0x7fa4100b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:55976/0 (socket says 172.21.15.111:55976) 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.907+0000 7fa40f7fe640 1 -- 172.21.15.111:0/2263436685 >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 msgr2=0x7fa4100726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:27.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.907+0000 7fa40f7fe640 1 --2- 172.21.15.111:0/2263436685 >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 0x7fa4100726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.907+0000 7fa40f7fe640 1 -- 172.21.15.111:0/2263436685 >> v1:172.21.15.153:6789/0 conn(0x7fa410144000 legacy=0x7fa4100b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.907+0000 7fa40f7fe640 1 -- 172.21.15.111:0/2263436685 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100b3ca0 con 0x7fa41006b390 2024-05-27T00:40:27.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.907+0000 7fa414a25640 1 --2- 172.21.15.111:0/2263436685 >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 0x7fa4100726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:27.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.907+0000 7fa40f7fe640 1 --2- 172.21.15.111:0/2263436685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4100cd740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4040097a0 tx=0x7fa40402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5187783be3e93a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa40effd640 1 -- 172.21.15.111:0/2263436685 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404037070 con 0x7fa41006b390 2024-05-27T00:40:27.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa40effd640 1 -- 172.21.15.111:0/2263436685 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa404032440 con 0x7fa41006b390 2024-05-27T00:40:27.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa40effd640 1 -- 172.21.15.111:0/2263436685 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404032810 con 0x7fa41006b390 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 -- 172.21.15.111:0/2263436685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 msgr2=0x7fa4100cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 --2- 172.21.15.111:0/2263436685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4100cd740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4040097a0 tx=0x7fa40402a180 comp rx=0 tx=0).stop 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 -- 172.21.15.111:0/2263436685 shutdown_connections 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 --2- 172.21.15.111:0/2263436685 >> v2:172.21.15.153:3300/0 conn(0x7fa4100cdc80 0x7fa4100726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 --2- 172.21.15.111:0/2263436685 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4100cd740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 -- 172.21.15.111:0/2263436685 >> 172.21.15.111:0/2263436685 conn(0x7fa4100b1c00 msgr2=0x7fa4100b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 -- 172.21.15.111:0/2263436685 shutdown_connections 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.908+0000 7fa416488640 1 -- 172.21.15.111:0/2263436685 wait complete. 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.909+0000 7fa416488640 1 Processor -- start 2024-05-27T00:40:27.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 -- start start 2024-05-27T00:40:27.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4101efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4100cdc80 0x7fa4101f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 0x7fa41013fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa410072d90 con 0x7fa410144000 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa410072c10 con 0x7fa4100cdc80 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa416488640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa410072f10 con 0x7fa41006b390 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa414a25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 0x7fa41013fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa414a25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 0x7fa41013fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40438/0 (socket says 172.21.15.111:40438) 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.910+0000 7fa414a25640 1 -- 172.21.15.111:0/2274203280 learned_addr learned my addr 172.21.15.111:0/2274203280 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa40f7fe640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4100cdc80 0x7fa4101f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa40ffff640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4101efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa414a25640 1 -- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 msgr2=0x7fa4101efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa414a25640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4101efac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa414a25640 1 -- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4100cdc80 msgr2=0x7fa4101f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa414a25640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4100cdc80 0x7fa4101f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa414a25640 1 -- 172.21.15.111:0/2274203280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4100ad7a0 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa40ffff640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4101efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa414a25640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 0x7fa41013fad0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa40800ef90 tx=0x7fa40800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40800cbe0 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40800cd80 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4101f00d0 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408003bb0 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.911+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4101f0690 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.912+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410068680 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.913+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa40801b030 con 0x7fa410144000 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.913+0000 7fa40cff9640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa3fc06be30 0x7fa3fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:27.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.913+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa40808af90 con 0x7fa410144000 2024-05-27T00:40:27.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.913+0000 7fa40ffff640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa3fc06be30 0x7fa3fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:27.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.914+0000 7fa40ffff640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa3fc06be30 0x7fa3fc06e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa400000f60 tx=0x7fa400009f70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:27.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:27.918+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa408022020 con 0x7fa410144000 2024-05-27T00:40:28.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.092+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa41006c800 con 0x7fa410144000 2024-05-27T00:40:28.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.093+0000 7fa40cff9640 1 -- 172.21.15.111:0/2274203280 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7fa4100ad7a0 con 0x7fa410144000 2024-05-27T00:40:28.094 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:28.094 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2024-05-27T00:40:28.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-05-27T00:40:28.094 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2024-05-27T00:40:28.095 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2024-05-27T00:40:28.095 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-05-27T00:40:28.095 INFO:teuthology.orchestra.run.smithi111.stdout:] 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa3fc06be30 msgr2=0x7fa3fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa3fc06be30 0x7fa3fc06e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa400000f60 tx=0x7fa400009f70 comp rx=0 tx=0).stop 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 msgr2=0x7fa41013fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 0x7fa41013fad0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa40800ef90 tx=0x7fa40800c490 comp rx=0 tx=0).stop 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 shutdown_connections 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa3fc06be30 0x7fa3fc06e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa410144000 0x7fa41013fad0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4100cdc80 0x7fa4101f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 --2- 172.21.15.111:0/2274203280 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa41006b390 0x7fa4101efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 >> 172.21.15.111:0/2274203280 conn(0x7fa4100b1c00 msgr2=0x7fa41006b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:28.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 shutdown_connections 2024-05-27T00:40:28.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.095+0000 7fa416488640 1 -- 172.21.15.111:0/2274203280 wait complete. 2024-05-27T00:40:28.107 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T00:40:28.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T00:40:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.210+0000 7f3775ad2640 1 Processor -- start 2024-05-27T00:40:28.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.210+0000 7f3775ad2640 1 -- start start 2024-05-27T00:40:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f3775ad2640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3770144000 0x7f37700b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f3775ad2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 0x7f3770072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f3775ad2640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3770073020 con 0x7f377006b390 2024-05-27T00:40:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f3775ad2640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f37700b42f0 con 0x7f3770144000 2024-05-27T00:40:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f3775ad2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f37700b3440 con 0x7f37700726d0 2024-05-27T00:40:28.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376ffff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 0x7f3770072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3770144000 0x7f37700b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3770144000 0x7f37700b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:39396/0 (socket says 172.21.15.111:39396) 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376ffff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 0x7f3770072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:39378/0 (socket says 172.21.15.111:39378) 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 -- 172.21.15.111:0/2582636113 learned_addr learned my addr 172.21.15.111:0/2582636113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 -- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 msgr2=0x7f3770072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 --2- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 0x7f3770072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 -- 172.21.15.111:0/2582636113 >> v1:172.21.15.153:6789/0 conn(0x7f377006b390 legacy=0x7f37700cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376f7fe640 1 -- 172.21.15.111:0/2582636113 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37700b36e0 con 0x7f3770144000 2024-05-27T00:40:28.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.211+0000 7f376ffff640 1 --2- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 0x7f3770072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:28.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.212+0000 7f376f7fe640 1 --2- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3770144000 0x7f37700b2f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3760009870 tx=0x7f376002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6f9befab12655d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.212+0000 7f376e7fc640 1 -- 172.21.15.111:0/2582636113 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760037070 con 0x7f3770144000 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.212+0000 7f376e7fc640 1 -- 172.21.15.111:0/2582636113 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3760032440 con 0x7f3770144000 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.212+0000 7f376e7fc640 1 -- 172.21.15.111:0/2582636113 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760032810 con 0x7f3770144000 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 -- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3770144000 msgr2=0x7f37700b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 --2- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3770144000 0x7f37700b2f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3760009870 tx=0x7f376002a270 comp rx=0 tx=0).stop 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 -- 172.21.15.111:0/2582636113 shutdown_connections 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 --2- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f37700726d0 0x7f3770072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 --2- 172.21.15.111:0/2582636113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3770144000 0x7f37700b2f00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 -- 172.21.15.111:0/2582636113 >> 172.21.15.111:0/2582636113 conn(0x7f37700b1c00 msgr2=0x7f37700b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 -- 172.21.15.111:0/2582636113 shutdown_connections 2024-05-27T00:40:28.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.213+0000 7f3775ad2640 1 -- 172.21.15.111:0/2582636113 wait complete. 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.214+0000 7f3775ad2640 1 Processor -- start 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.214+0000 7f3775ad2640 1 -- start start 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f3775ad2640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f377006b390 0x7f37700ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f3775ad2640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37700726d0 0x7f37701e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f3775ad2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 0x7f377013d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f3775ad2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f37700b3f80 con 0x7f3770144000 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f3775ad2640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f37700b3ca0 con 0x7f37700726d0 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f3775ad2640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f37700cde50 con 0x7f377006b390 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 0x7f377013d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 0x7f377013d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40448/0 (socket says 172.21.15.111:40448) 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 -- 172.21.15.111:0/3741477677 learned_addr learned my addr 172.21.15.111:0/3741477677 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376effd640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37700726d0 0x7f37701e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376f7fe640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f377006b390 0x7f37700ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 -- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f377006b390 msgr2=0x7f37700ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f377006b390 0x7f37700ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 -- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37700726d0 msgr2=0x7f37701e06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37700726d0 0x7f37701e06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376ffff640 1 -- 172.21.15.111:0/3741477677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377013de00 con 0x7f3770144000 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376effd640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37700726d0 0x7f37701e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.215+0000 7f376f7fe640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f377006b390 0x7f37700ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.216+0000 7f376ffff640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 0x7f377013d8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f376800efc0 tx=0x7f3768009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.216+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768018030 con 0x7f3770144000 2024-05-27T00:40:28.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.216+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376800cde0 con 0x7f3770144000 2024-05-27T00:40:28.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.216+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37700ad8d0 con 0x7f3770144000 2024-05-27T00:40:28.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.216+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768003bb0 con 0x7f3770144000 2024-05-27T00:40:28.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.216+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37700ade90 con 0x7f3770144000 2024-05-27T00:40:28.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.217+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3770068680 con 0x7f3770144000 2024-05-27T00:40:28.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.217+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3768005d10 con 0x7f3770144000 2024-05-27T00:40:28.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.218+0000 7f3757fff640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375c06be60 0x7f375c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.218+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f376808b670 con 0x7f3770144000 2024-05-27T00:40:28.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.218+0000 7f376f7fe640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375c06be60 0x7f375c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.218+0000 7f376f7fe640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375c06be60 0x7f375c06e320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3760034000 tx=0x7f376002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:28.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.222+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3768022070 con 0x7f3770144000 2024-05-27T00:40:28.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.396+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f37700afa70 con 0x7f3770144000 2024-05-27T00:40:28.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.816+0000 7f3757fff640 1 -- 172.21.15.111:0/3741477677 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f3768010040 con 0x7f3770144000 2024-05-27T00:40:28.817 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2024-05-27T00:40:28.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375c06be60 msgr2=0x7f375c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375c06be60 0x7f375c06e320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3760034000 tx=0x7f376002e000 comp rx=0 tx=0).stop 2024-05-27T00:40:28.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 msgr2=0x7f377013d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 0x7f377013d8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f376800efc0 tx=0x7f3768009f90 comp rx=0 tx=0).stop 2024-05-27T00:40:28.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 shutdown_connections 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f375c06be60 0x7f375c06e320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3770144000 0x7f377013d8c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37700726d0 0x7f37701e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 --2- 172.21.15.111:0/3741477677 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f377006b390 0x7f37700ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 >> 172.21.15.111:0/3741477677 conn(0x7f37700b1c00 msgr2=0x7f37700cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 shutdown_connections 2024-05-27T00:40:28.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.818+0000 7f3775ad2640 1 -- 172.21.15.111:0/3741477677 wait complete. 2024-05-27T00:40:28.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T00:40:28.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.940+0000 7f481102c640 1 Processor -- start 2024-05-27T00:40:28.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.940+0000 7f481102c640 1 -- start start 2024-05-27T00:40:28.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f481102c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f480c06b390 0x7f480c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f481102c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c0cdc80 0x7f480c0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f481102c640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480c0b36e0 con 0x7f480c144000 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f481102c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f480c0b42f0 con 0x7f480c06b390 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f481102c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f480c0b39c0 con 0x7f480c0cdc80 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f480c06b390 0x7f480c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f480c06b390 0x7f480c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40460/0 (socket says 172.21.15.111:40460) 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 -- 172.21.15.111:0/2309914577 learned_addr learned my addr 172.21.15.111:0/2309914577 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480b59e640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c0cdc80 0x7f480c0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 -- 172.21.15.111:0/2309914577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c0cdc80 msgr2=0x7f480c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c0cdc80 0x7f480c0726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 -- 172.21.15.111:0/2309914577 >> v1:172.21.15.153:6789/0 conn(0x7f480c144000 legacy=0x7f480c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480259c640 1 -- 172.21.15.111:0/2309914577 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c0b3ca0 con 0x7f480c06b390 2024-05-27T00:40:28.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.941+0000 7f480b59e640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c0cdc80 0x7f480c0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.942+0000 7f480259c640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c06b390 0x7f480c0cd740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009870 tx=0x7f47f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b9c8a36163092c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.942+0000 7f480a59c640 1 -- 172.21.15.111:0/2309914577 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4037070 con 0x7f480c06b390 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.942+0000 7f480a59c640 1 -- 172.21.15.111:0/2309914577 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f4032440 con 0x7f480c06b390 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.942+0000 7f480a59c640 1 -- 172.21.15.111:0/2309914577 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4032810 con 0x7f480c06b390 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 -- 172.21.15.111:0/2309914577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c06b390 msgr2=0x7f480c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c06b390 0x7f480c0cd740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009870 tx=0x7f47f402a270 comp rx=0 tx=0).stop 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 -- 172.21.15.111:0/2309914577 shutdown_connections 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c0cdc80 0x7f480c0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 --2- 172.21.15.111:0/2309914577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c06b390 0x7f480c0cd740 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 -- 172.21.15.111:0/2309914577 >> 172.21.15.111:0/2309914577 conn(0x7f480c0b1c00 msgr2=0x7f480c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:28.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 -- 172.21.15.111:0/2309914577 shutdown_connections 2024-05-27T00:40:28.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.943+0000 7f481102c640 1 -- 172.21.15.111:0/2309914577 wait complete. 2024-05-27T00:40:28.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.944+0000 7f481102c640 1 Processor -- start 2024-05-27T00:40:28.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.944+0000 7f481102c640 1 -- start start 2024-05-27T00:40:28.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f481102c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c06b390 0x7f480c1f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f481102c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f480c0cdc80 0x7f480c1f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f481102c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 0x7f480c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f481102c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f480c072d90 con 0x7f480c144000 2024-05-27T00:40:28.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f481102c640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f480c072c10 con 0x7f480c0cdc80 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f481102c640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f480c072f10 con 0x7f480c06b390 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f480b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 0x7f480c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f480b59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 0x7f480c15dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40464/0 (socket says 172.21.15.111:40464) 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f480b59e640 1 -- 172.21.15.111:0/3475947910 learned_addr learned my addr 172.21.15.111:0/3475947910 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f480ad9d640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c06b390 0x7f480c1f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.945+0000 7f480259c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f480c0cdc80 0x7f480c1f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f480b59e640 1 -- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c06b390 msgr2=0x7f480c1f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f480b59e640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c06b390 0x7f480c1f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f480b59e640 1 -- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f480c0cdc80 msgr2=0x7f480c1f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f480b59e640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f480c0cdc80 0x7f480c1f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f480b59e640 1 -- 172.21.15.111:0/3475947910 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1f4e70 con 0x7f480c144000 2024-05-27T00:40:28.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f480b59e640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 0x7f480c15dab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4804007e70 tx=0x7f480400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480401a070 con 0x7f480c144000 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4804015600 con 0x7f480c144000 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c15c770 con 0x7f480c144000 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040159f0 con 0x7f480c144000 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.946+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c15ccf0 con 0x7f480c144000 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.947+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c068680 con 0x7f480c144000 2024-05-27T00:40:28.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.948+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f480401d070 con 0x7f480c144000 2024-05-27T00:40:28.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.948+0000 7f4803fff640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47ec06be30 0x7f47ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:28.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.948+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f480408afe0 con 0x7f480c144000 2024-05-27T00:40:28.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.948+0000 7f480ad9d640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47ec06be30 0x7f47ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:28.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.949+0000 7f480ad9d640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47ec06be30 0x7f47ec06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0016e0 tx=0x7f47fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:28.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:28.953+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f480c1f4e70 con 0x7f480c144000 2024-05-27T00:40:29.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.129+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f480c0b1660 con 0x7f480c144000 2024-05-27T00:40:29.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.818+0000 7f4803fff640 1 -- 172.21.15.111:0/3475947910 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f4804010070 con 0x7f480c144000 2024-05-27T00:40:29.820 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2024-05-27T00:40:29.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.822+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47ec06be30 msgr2=0x7f47ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:29.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.822+0000 7f481102c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47ec06be30 0x7f47ec06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0016e0 tx=0x7f47fc009040 comp rx=0 tx=0).stop 2024-05-27T00:40:29.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.822+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 msgr2=0x7f480c15dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:29.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.822+0000 7f481102c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 0x7f480c15dab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4804007e70 tx=0x7f480400d4a0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.823+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 shutdown_connections 2024-05-27T00:40:29.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.823+0000 7f481102c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f47ec06be30 0x7f47ec06e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.823+0000 7f481102c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f480c144000 0x7f480c15dab0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.823+0000 7f481102c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f480c0cdc80 0x7f480c1f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.823+0000 7f481102c640 1 --2- 172.21.15.111:0/3475947910 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f480c06b390 0x7f480c1f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.823+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 >> 172.21.15.111:0/3475947910 conn(0x7f480c0b1c00 msgr2=0x7f480c071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:29.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.825+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 shutdown_connections 2024-05-27T00:40:29.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.825+0000 7f481102c640 1 -- 172.21.15.111:0/3475947910 wait complete. 2024-05-27T00:40:29.848 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T00:40:29.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.951+0000 7f861c361640 1 Processor -- start 2024-05-27T00:40:29.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.951+0000 7f861c361640 1 -- start start 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.952+0000 7f861c361640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f861413f340 0x7f86140cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.952+0000 7f861c361640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.952+0000 7f861c361640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8614142650 con 0x7f861413ef60 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.952+0000 7f861c361640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f86140cf210 con 0x7f861413f340 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.952+0000 7f861c361640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f86140cebc0 con 0x7f86140cce00 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f861413f340 0x7f86140cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f861413f340 0x7f86140cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40478/0 (socket says 172.21.15.111:40478) 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 -- 172.21.15.111:0/3519533042 learned_addr learned my addr 172.21.15.111:0/3519533042 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:29.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f861a8fe640 1 --2- 172.21.15.111:0/3519533042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:29.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 -- 172.21.15.111:0/3519533042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 msgr2=0x7f8614141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:29.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 --2- 172.21.15.111:0/3519533042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 -- 172.21.15.111:0/3519533042 >> v1:172.21.15.153:6789/0 conn(0x7f861413ef60 legacy=0x7f86140cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:29.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.953+0000 7f86198fc640 1 -- 172.21.15.111:0/3519533042 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86140ceea0 con 0x7f861413f340 2024-05-27T00:40:29.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f86198fc640 1 --2- 172.21.15.111:0/3519533042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f86140cc8c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8604009870 tx=0x7f860402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dd2ad114efb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:29.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f86190fb640 1 -- 172.21.15.111:0/3519533042 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604037070 con 0x7f861413f340 2024-05-27T00:40:29.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f86190fb640 1 -- 172.21.15.111:0/3519533042 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8604032440 con 0x7f861413f340 2024-05-27T00:40:29.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f86190fb640 1 -- 172.21.15.111:0/3519533042 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604032810 con 0x7f861413f340 2024-05-27T00:40:29.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 -- 172.21.15.111:0/3519533042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 msgr2=0x7f86140cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 --2- 172.21.15.111:0/3519533042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f86140cc8c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8604009870 tx=0x7f860402a270 comp rx=0 tx=0).stop 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 -- 172.21.15.111:0/3519533042 shutdown_connections 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 --2- 172.21.15.111:0/3519533042 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 --2- 172.21.15.111:0/3519533042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f86140cc8c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 -- 172.21.15.111:0/3519533042 >> 172.21.15.111:0/3519533042 conn(0x7f86140b1c00 msgr2=0x7f86140b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 -- 172.21.15.111:0/3519533042 shutdown_connections 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.954+0000 7f861c361640 1 -- 172.21.15.111:0/3519533042 wait complete. 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 Processor -- start 2024-05-27T00:40:29.957 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 -- start start 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861413ef60 0x7f861413e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f8614157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8614141fe0 con 0x7f861413f340 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8614141e60 con 0x7f861413ef60 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861c361640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8614142160 con 0x7f86140cce00 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861a8fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f8614157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861a8fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f8614157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40482/0 (socket says 172.21.15.111:40482) 2024-05-27T00:40:29.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.956+0000 7f861a8fe640 1 -- 172.21.15.111:0/2478312363 learned_addr learned my addr 172.21.15.111:0/2478312363 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f86198fc640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861413ef60 0x7f861413e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a0fd640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a8fe640 1 -- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 msgr2=0x7f8614145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a8fe640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a8fe640 1 -- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861413ef60 msgr2=0x7f861413e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a8fe640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861413ef60 0x7f861413e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a8fe640 1 -- 172.21.15.111:0/2478312363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861413ed80 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861a8fe640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f8614157e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f860c009d30 tx=0x7f860c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c018020 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c014480 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8614071160 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c014870 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.957+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8614071400 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.958+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8614068680 con 0x7f861413f340 2024-05-27T00:40:29.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.959+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f860c005d10 con 0x7f861413f340 2024-05-27T00:40:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.959+0000 7f860affd640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85f806be30 0x7f85f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.959+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f860c08b9e0 con 0x7f861413f340 2024-05-27T00:40:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.959+0000 7f861a0fd640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85f806be30 0x7f85f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:29.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.960+0000 7f861a0fd640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85f806be30 0x7f85f806e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8610007fd0 tx=0x7f8610009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:29.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:29.964+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f860c022070 con 0x7f861413f340 2024-05-27T00:40:30.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.174+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f861415e580 con 0x7f861413f340 2024-05-27T00:40:30.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.827+0000 7f860affd640 1 -- 172.21.15.111:0/2478312363 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f861413ed80 con 0x7f861413f340 2024-05-27T00:40:30.829 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 5 and data pool 6 2024-05-27T00:40:30.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.830+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85f806be30 msgr2=0x7f85f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:30.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85f806be30 0x7f85f806e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8610007fd0 tx=0x7f8610009040 comp rx=0 tx=0).stop 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 msgr2=0x7f8614157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f8614157e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f860c009d30 tx=0x7f860c00c490 comp rx=0 tx=0).stop 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 shutdown_connections 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f85f806be30 0x7f85f806e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f861413f340 0x7f8614157e90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f861413ef60 0x7f861413e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 --2- 172.21.15.111:0/2478312363 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f86140cce00 0x7f8614145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.831+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 >> 172.21.15.111:0/2478312363 conn(0x7f86140b1c00 msgr2=0x7f8614160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:30.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.834+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 shutdown_connections 2024-05-27T00:40:30.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.836+0000 7f861c361640 1 -- 172.21.15.111:0/2478312363 wait complete. 2024-05-27T00:40:30.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:30.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.969+0000 7f5e441ee640 1 Processor -- start 2024-05-27T00:40:30.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.969+0000 7f5e441ee640 1 -- start start 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e441ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c144000 0x7f5e3c0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e441ee640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 0x7f5e3c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e441ee640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e3c0b3440 con 0x7f5e3c0cdc80 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e441ee640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3c0b42f0 con 0x7f5e3c06b390 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e441ee640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b36e0 con 0x7f5e3c144000 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e41789640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 0x7f5e3c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e41f8a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c144000 0x7f5e3c0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:30.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e41789640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 0x7f5e3c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:50010/0 (socket says 172.21.15.111:50010) 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e41789640 1 -- 172.21.15.111:0/2463238743 learned_addr learned my addr 172.21.15.111:0/2463238743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.970+0000 7f5e4278b640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5e3c0cdc80 0x7f5e3c0726d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:41616/0 (socket says 172.21.15.111:41616) 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.971+0000 7f5e41f8a640 1 -- 172.21.15.111:0/2463238743 >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 msgr2=0x7f5e3c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.971+0000 7f5e41f8a640 1 --2- 172.21.15.111:0/2463238743 >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 0x7f5e3c0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.971+0000 7f5e41f8a640 1 -- 172.21.15.111:0/2463238743 >> v1:172.21.15.153:6789/0 conn(0x7f5e3c0cdc80 legacy=0x7f5e3c0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.971+0000 7f5e41f8a640 1 -- 172.21.15.111:0/2463238743 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0b39c0 con 0x7f5e3c144000 2024-05-27T00:40:30.973 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.971+0000 7f5e41789640 1 --2- 172.21.15.111:0/2463238743 >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 0x7f5e3c0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:30.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.971+0000 7f5e41f8a640 1 --2- 172.21.15.111:0/2463238743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c144000 0x7f5e3c0b2f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009d90 tx=0x7f5e2c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb186d481516154 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:30.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e40f88640 1 -- 172.21.15.111:0/2463238743 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c007740 con 0x7f5e3c144000 2024-05-27T00:40:30.976 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e40f88640 1 -- 172.21.15.111:0/2463238743 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0078e0 con 0x7f5e3c144000 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e40f88640 1 -- 172.21.15.111:0/2463238743 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c007cb0 con 0x7f5e3c144000 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 -- 172.21.15.111:0/2463238743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c144000 msgr2=0x7f5e3c0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 --2- 172.21.15.111:0/2463238743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c144000 0x7f5e3c0b2f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009d90 tx=0x7f5e2c02a660 comp rx=0 tx=0).stop 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 -- 172.21.15.111:0/2463238743 shutdown_connections 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 --2- 172.21.15.111:0/2463238743 >> v2:172.21.15.153:3300/0 conn(0x7f5e3c06b390 0x7f5e3c0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 --2- 172.21.15.111:0/2463238743 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c144000 0x7f5e3c0b2f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 -- 172.21.15.111:0/2463238743 >> 172.21.15.111:0/2463238743 conn(0x7f5e3c0b1c00 msgr2=0x7f5e3c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 -- 172.21.15.111:0/2463238743 shutdown_connections 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.972+0000 7f5e441ee640 1 -- 172.21.15.111:0/2463238743 wait complete. 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.973+0000 7f5e441ee640 1 Processor -- start 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.973+0000 7f5e441ee640 1 -- start start 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e441ee640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c06b390 0x7f5e3c138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:30.977 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e441ee640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e3c0cdc80 0x7f5e3c0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e441ee640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 0x7f5e3c0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e441ee640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c072de0 con 0x7f5e3c144000 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e441ee640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b3f80 con 0x7f5e3c0cdc80 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e441ee640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c072f60 con 0x7f5e3c06b390 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e4278b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 0x7f5e3c0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e4278b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 0x7f5e3c0c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59368/0 (socket says 172.21.15.111:59368) 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e4278b640 1 -- 172.21.15.111:0/1152324704 learned_addr learned my addr 172.21.15.111:0/1152324704 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e41f8a640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c06b390 0x7f5e3c138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.974+0000 7f5e41789640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e3c0cdc80 0x7f5e3c0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e4278b640 1 -- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c06b390 msgr2=0x7f5e3c138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e4278b640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c06b390 0x7f5e3c138ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e4278b640 1 -- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e3c0cdc80 msgr2=0x7f5e3c0c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:30.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e4278b640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e3c0cdc80 0x7f5e3c0c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e4278b640 1 -- 172.21.15.111:0/1152324704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0abf20 con 0x7f5e3c144000 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e4278b640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 0x7f5e3c0c53b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c430 tx=0x7f5e3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e340074e0 con 0x7f5e3c144000 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34007680 con 0x7f5e3c144000 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c0abc40 con 0x7f5e3c144000 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e340049e0 con 0x7f5e3c144000 2024-05-27T00:40:30.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.975+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c133670 con 0x7f5e3c144000 2024-05-27T00:40:30.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.976+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c0b5010 con 0x7f5e3c144000 2024-05-27T00:40:30.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.976+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e3401a020 con 0x7f5e3c144000 2024-05-27T00:40:30.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.977+0000 7f5e3a7fc640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5e2806be30 0x7f5e2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:30.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.977+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5e3408a600 con 0x7f5e3c144000 2024-05-27T00:40:30.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.977+0000 7f5e41f8a640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5e2806be30 0x7f5e2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:30.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.978+0000 7f5e41f8a640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5e2806be30 0x7f5e2806e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c034000 tx=0x7f5e2c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:30.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:30.982+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5e3c0b5010 con 0x7f5e3c144000 2024-05-27T00:40:31.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.157+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e3c068680 con 0x7f5e3c144000 2024-05-27T00:40:31.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.158+0000 7f5e3a7fc640 1 -- 172.21.15.111:0/1152324704 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17003 (secure 0 0 0) 0x7f5e340587d0 con 0x7f5e3c144000 2024-05-27T00:40:31.160 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:31.160 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":38,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:30.175718+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:40:28.415554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-27T00:40:29.147789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5e2806be30 msgr2=0x7f5e2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5e2806be30 0x7f5e2806e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c034000 tx=0x7f5e2c02f000 comp rx=0 tx=0).stop 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 msgr2=0x7f5e3c0c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 0x7f5e3c0c53b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c430 tx=0x7f5e3400c8f0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 shutdown_connections 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f5e2806be30 0x7f5e2806e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e3c144000 0x7f5e3c0c53b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e3c0cdc80 0x7f5e3c0c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 --2- 172.21.15.111:0/1152324704 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5e3c06b390 0x7f5e3c138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 >> 172.21.15.111:0/1152324704 conn(0x7f5e3c0b1c00 msgr2=0x7f5e3c071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:31.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 shutdown_connections 2024-05-27T00:40:31.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.161+0000 7f5e441ee640 1 -- 172.21.15.111:0/1152324704 wait complete. 2024-05-27T00:40:31.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.272+0000 7fa6ad89e640 1 Processor -- start 2024-05-27T00:40:31.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.272+0000 7fa6ad89e640 1 -- start start 2024-05-27T00:40:31.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6ad89e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6a80b3630 0x7fa6a80b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6ad89e640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a806cc70 con 0x7fa6a8162560 2024-05-27T00:40:31.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6ad89e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a80d1e50 con 0x7fa6a80b77e0 2024-05-27T00:40:31.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6ad89e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a80d2130 con 0x7fa6a80b3630 2024-05-27T00:40:31.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6a77fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa6a8162560 0x7fa6a80d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58922/0 (socket says 172.21.15.111:58922) 2024-05-27T00:40:31.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6a77fe640 1 -- 172.21.15.111:0/2249375625 learned_addr learned my addr 172.21.15.111:0/2249375625 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:31.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006759944 0 0) 0x7fa6a806cc70 con 0x7fa6a8162560 2024-05-27T00:40:31.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa694003520 con 0x7fa6a8162560 2024-05-27T00:40:31.278 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.273+0000 7fa6a6ffd640 1 --2- 172.21.15.111:0/2249375625 >> v2:172.21.15.153:3300/0 conn(0x7fa6a80b3630 0x7fa6a80b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 771180155 0 0) 0x7fa694003520 con 0x7fa6a8162560 2024-05-27T00:40:31.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 >> v2:172.21.15.153:3300/0 conn(0x7fa6a80b3630 msgr2=0x7fa6a80b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:31.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 --2- 172.21.15.111:0/2249375625 >> v2:172.21.15.153:3300/0 conn(0x7fa6a80b3630 0x7fa6a80b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 >> v1:172.21.15.153:6789/0 conn(0x7fa6a80b77e0 legacy=0x7fa6a8161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a80d2410 con 0x7fa6a8162560 2024-05-27T00:40:31.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fa6a00019f0 con 0x7fa6a8162560 2024-05-27T00:40:31.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a6ffd640 1 --2- 172.21.15.111:0/2249375625 >> v2:172.21.15.153:3300/0 conn(0x7fa6a80b3630 0x7fa6a80b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:31.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 >> v1:172.21.15.111:6789/0 conn(0x7fa6a8162560 legacy=0x7fa6a80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6940039b0 0x7fa694003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694004800 0x7fa694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa69400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80d2130 con 0x7fa6940051c0 2024-05-27T00:40:31.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa69400c110 con 0x7fa694004800 2024-05-27T00:40:31.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa69400c310 con 0x7fa6940039b0 2024-05-27T00:40:31.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a77fe640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa69400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa69e7fc640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694004800 0x7fa694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a6ffd640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6940039b0 0x7fa694003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a77fe640 1 -- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6940039b0 msgr2=0x7fa694003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:31.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a77fe640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6940039b0 0x7fa694003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a77fe640 1 -- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694004800 msgr2=0x7fa694004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:31.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a77fe640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694004800 0x7fa694004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa6a77fe640 1 -- 172.21.15.111:0/2249375625 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694003520 con 0x7fa6940051c0 2024-05-27T00:40:31.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.274+0000 7fa69e7fc640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa694004800 0x7fa694004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:31.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6a77fe640 1 -- 172.21.15.111:0/2249375625 reap_dead start 2024-05-27T00:40:31.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6a77fe640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa69400b9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa6a808d4a0 tx=0x7fa694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4c3ee57e3cfed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:31.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000a480 con 0x7fa6940051c0 2024-05-27T00:40:31.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a002f070 con 0x7fa6940051c0 2024-05-27T00:40:31.283 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6a67fc640 1 -- 172.21.15.111:0/2249375625 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a002bbf0 con 0x7fa6940051c0 2024-05-27T00:40:31.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 -- 172.21.15.111:0/2249375625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 msgr2=0x7fa69400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa69400b9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa6a808d4a0 tx=0x7fa694005830 comp rx=0 tx=0).stop 2024-05-27T00:40:31.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 -- 172.21.15.111:0/2249375625 shutdown_connections 2024-05-27T00:40:31.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/2249375625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa69400b9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 -- 172.21.15.111:0/2249375625 >> 172.21.15.111:0/2249375625 conn(0x7fa6a80d42c0 msgr2=0x7fa6a80d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:31.284 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 -- 172.21.15.111:0/2249375625 shutdown_connections 2024-05-27T00:40:31.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.275+0000 7fa6ad89e640 1 -- 172.21.15.111:0/2249375625 wait complete. 2024-05-27T00:40:31.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.277+0000 7fa6ad89e640 1 Processor -- start 2024-05-27T00:40:31.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.277+0000 7fa6ad89e640 1 -- start start 2024-05-27T00:40:31.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa6a814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6a80ca2e0 0x7fa6a80ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6a80b8a10 0x7fa6a80b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80b7420 con 0x7fa6940051c0 2024-05-27T00:40:31.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a80b72a0 con 0x7fa6a80ca2e0 2024-05-27T00:40:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a80b75a0 con 0x7fa6a80b8a10 2024-05-27T00:40:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa6a814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa6a814f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59376/0 (socket says 172.21.15.111:59376) 2024-05-27T00:40:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 -- 172.21.15.111:0/761430269 learned_addr learned my addr 172.21.15.111:0/761430269 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:31.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa69e7fc640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6a80ca2e0 0x7fa6a80ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a77fe640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6a80b8a10 0x7fa6a80b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 -- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6a80b8a10 msgr2=0x7fa6a80b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6a80b8a10 0x7fa6a80b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 -- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6a80ca2e0 msgr2=0x7fa6a80ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6a80ca2e0 0x7fa6a80ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 -- 172.21.15.111:0/761430269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a80cb2e0 con 0x7fa6940051c0 2024-05-27T00:40:31.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a6ffd640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa6a814f430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6980115c0 tx=0x7fa698011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:31.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698019070 con 0x7fa6940051c0 2024-05-27T00:40:31.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa698004220 con 0x7fa6940051c0 2024-05-27T00:40:31.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69800f040 con 0x7fa6940051c0 2024-05-27T00:40:31.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a80cb580 con 0x7fa6940051c0 2024-05-27T00:40:31.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.278+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a80cbb40 con 0x7fa6940051c0 2024-05-27T00:40:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.279+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a8068680 con 0x7fa6940051c0 2024-05-27T00:40:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.280+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa698015670 con 0x7fa6940051c0 2024-05-27T00:40:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.280+0000 7fa6a4ff9640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa68806be30 0x7fa68806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.281+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa69808b080 con 0x7fa6940051c0 2024-05-27T00:40:31.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.281+0000 7fa69e7fc640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa68806be30 0x7fa68806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.281+0000 7fa69e7fc640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa68806be30 0x7fa68806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa690000c00 tx=0x7fa690009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:31.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.285+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa6980592e0 con 0x7fa6940051c0 2024-05-27T00:40:31.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.460+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6a80cef10 con 0x7fa6940051c0 2024-05-27T00:40:31.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.461+0000 7fa6a4ff9640 1 -- 172.21.15.111:0/761430269 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17003 (secure 0 0 0) 0x7fa69805d5e0 con 0x7fa6940051c0 2024-05-27T00:40:31.462 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:31.462 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":38,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:30.175718+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:40:28.415554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-27T00:40:29.147789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa68806be30 msgr2=0x7fa68806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa68806be30 0x7fa68806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa690000c00 tx=0x7fa690009040 comp rx=0 tx=0).stop 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 msgr2=0x7fa6a814f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa6a814f430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6980115c0 tx=0x7fa698011a80 comp rx=0 tx=0).stop 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 shutdown_connections 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fa68806be30 0x7fa68806e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fa6a80b8a10 0x7fa6a80b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6a80ca2e0 0x7fa6a80ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 --2- 172.21.15.111:0/761430269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6940051c0 0x7fa6a814f430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 >> 172.21.15.111:0/761430269 conn(0x7fa6a80d42c0 msgr2=0x7fa6a80d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:31.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 shutdown_connections 2024-05-27T00:40:31.465 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.463+0000 7fa6ad89e640 1 -- 172.21.15.111:0/761430269 wait complete. 2024-05-27T00:40:31.475 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-27T00:40:31.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.577+0000 7f1bb4109640 1 Processor -- start 2024-05-27T00:40:31.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.577+0000 7f1bb4109640 1 -- start start 2024-05-27T00:40:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb4109640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1bac06b390 0x7f1bac0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb4109640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1bac0cdc80 0x7f1bac0726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb4109640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bac0b36e0 con 0x7f1bac144000 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb4109640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f1bac0b42f0 con 0x7f1bac06b390 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb4109640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f1bac0b39c0 con 0x7f1bac0cdc80 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb16a4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1bac06b390 0x7f1bac0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb1ea5640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1bac144000 0x7f1bac0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58932/0 (socket says 172.21.15.111:58932) 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb1ea5640 1 -- 172.21.15.111:0/3629257959 learned_addr learned my addr 172.21.15.111:0/3629257959 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb16a4640 1 -- 172.21.15.111:0/3629257959 >> v2:172.21.15.153:3300/0 conn(0x7f1bac0cdc80 msgr2=0x7f1bac0726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb26a6640 1 --2- 172.21.15.111:0/3629257959 >> v2:172.21.15.153:3300/0 conn(0x7f1bac0cdc80 0x7f1bac0726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb0ea3640 1 -- 172.21.15.111:0/3629257959 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583819336 0 0) 0x7f1bac0b36e0 con 0x7f1bac144000 2024-05-27T00:40:31.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb16a4640 1 --2- 172.21.15.111:0/3629257959 >> v2:172.21.15.153:3300/0 conn(0x7f1bac0cdc80 0x7f1bac0726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb16a4640 1 -- 172.21.15.111:0/3629257959 >> v1:172.21.15.111:6789/0 conn(0x7f1bac144000 legacy=0x7f1bac0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.578+0000 7f1bb16a4640 1 -- 172.21.15.111:0/3629257959 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac0b3ca0 con 0x7f1bac06b390 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.579+0000 7f1bb16a4640 1 --2- 172.21.15.111:0/3629257959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac06b390 0x7f1bac0cd740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1bac06e340 tx=0x7f1b9c02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4e664b3705b83b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.579+0000 7f1bb0ea3640 1 -- 172.21.15.111:0/3629257959 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c032770 con 0x7f1bac06b390 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.579+0000 7f1bb0ea3640 1 -- 172.21.15.111:0/3629257959 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9c038070 con 0x7f1bac06b390 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.579+0000 7f1bb0ea3640 1 -- 172.21.15.111:0/3629257959 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c032b30 con 0x7f1bac06b390 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 -- 172.21.15.111:0/3629257959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac06b390 msgr2=0x7f1bac0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3629257959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac06b390 0x7f1bac0cd740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1bac06e340 tx=0x7f1b9c02c870 comp rx=0 tx=0).stop 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 -- 172.21.15.111:0/3629257959 shutdown_connections 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3629257959 >> v2:172.21.15.153:3300/0 conn(0x7f1bac0cdc80 0x7f1bac0726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3629257959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac06b390 0x7f1bac0cd740 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 -- 172.21.15.111:0/3629257959 >> 172.21.15.111:0/3629257959 conn(0x7f1bac0b1c00 msgr2=0x7f1bac0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 -- 172.21.15.111:0/3629257959 shutdown_connections 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.580+0000 7f1bb4109640 1 -- 172.21.15.111:0/3629257959 wait complete. 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.581+0000 7f1bb4109640 1 Processor -- start 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.581+0000 7f1bb4109640 1 -- start start 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb4109640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bac06b390 0x7f1bac159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb4109640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bac0cdc80 0x7f1bac132710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb4109640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 0x7f1bac0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb4109640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac072d90 con 0x7f1bac144000 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb4109640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac072c10 con 0x7f1bac06b390 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb4109640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f1bac072f10 con 0x7f1bac0cdc80 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 0x7f1bac0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 0x7f1bac0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59392/0 (socket says 172.21.15.111:59392) 2024-05-27T00:40:31.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 -- 172.21.15.111:0/3028853475 learned_addr learned my addr 172.21.15.111:0/3028853475 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 -- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bac0cdc80 msgr2=0x7f1bac132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bac0cdc80 0x7f1bac132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 -- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bac06b390 msgr2=0x7f1bac159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb1ea5640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bac06b390 0x7f1bac159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bac06b390 0x7f1bac159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 -- 172.21.15.111:0/3028853475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac1331d0 con 0x7f1bac144000 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1bb26a6640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 0x7f1bac0ce060 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400cd70 tx=0x7f1ba400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.582+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4003b90 con 0x7f1bac144000 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.583+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba4003d30 con 0x7f1bac144000 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.583+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac1334b0 con 0x7f1bac144000 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.583+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4004240 con 0x7f1bac144000 2024-05-27T00:40:31.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.583+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac133a70 con 0x7f1bac144000 2024-05-27T00:40:31.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.584+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac068680 con 0x7f1bac144000 2024-05-27T00:40:31.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.584+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ba40043e0 con 0x7f1bac144000 2024-05-27T00:40:31.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.584+0000 7f1ba27fc640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b9006be30 0x7f1b9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:31.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.584+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1ba408baf0 con 0x7f1bac144000 2024-05-27T00:40:31.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.585+0000 7f1bb1ea5640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b9006be30 0x7f1b9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:31.588 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.585+0000 7f1bb1ea5640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b9006be30 0x7f1b9006e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8000f60 tx=0x7f1ba8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:31.589 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.589+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ba405a310 con 0x7f1bac144000 2024-05-27T00:40:31.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:31.799+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1bac0714e0 con 0x7f1bac144000 2024-05-27T00:40:32.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.396+0000 7f1ba27fc640 1 -- 172.21.15.111:0/3028853475 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f1ba4059c10 con 0x7f1bac144000 2024-05-27T00:40:32.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b9006be30 msgr2=0x7f1b9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:32.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b9006be30 0x7f1b9006e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8000f60 tx=0x7f1ba8009040 comp rx=0 tx=0).stop 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 msgr2=0x7f1bac0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 0x7f1bac0ce060 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400cd70 tx=0x7f1ba400a660 comp rx=0 tx=0).stop 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 shutdown_connections 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f1b9006be30 0x7f1b9006e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bac144000 0x7f1bac0ce060 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f1bac0cdc80 0x7f1bac132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 --2- 172.21.15.111:0/3028853475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bac06b390 0x7f1bac159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 >> 172.21.15.111:0/3028853475 conn(0x7f1bac0b1c00 msgr2=0x7f1bac0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 shutdown_connections 2024-05-27T00:40:32.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.398+0000 7f1bb4109640 1 -- 172.21.15.111:0/3028853475 wait complete. 2024-05-27T00:40:32.411 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-27T00:40:32.411 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-27T00:40:32.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.515+0000 7f47378c1640 1 Processor -- start 2024-05-27T00:40:32.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.515+0000 7f47378c1640 1 -- start start 2024-05-27T00:40:32.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f47378c1640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f473006b390 0x7f47300cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:32.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f47378c1640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47300b36e0 con 0x7f4730144000 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f47378c1640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47300b39c0 con 0x7f47300cdc80 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f47378c1640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f47300b42f0 con 0x7f473006b390 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f4734e5c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f473006b390 0x7f47300cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f473565d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4730144000 0x7f47300b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58946/0 (socket says 172.21.15.111:58946) 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.516+0000 7f473565d640 1 -- 172.21.15.111:0/1254584707 learned_addr learned my addr 172.21.15.111:0/1254584707 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f4734e5c640 1 -- 172.21.15.111:0/1254584707 >> v1:172.21.15.153:6789/0 conn(0x7f47300cdc80 legacy=0x7f47300726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f4734e5c640 1 -- 172.21.15.111:0/1254584707 >> v1:172.21.15.111:6789/0 conn(0x7f4730144000 legacy=0x7f47300b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f472ffff640 1 -- 172.21.15.111:0/1254584707 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465017086 0 0) 0x7f47300b36e0 con 0x7f4730144000 2024-05-27T00:40:32.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f4734e5c640 1 -- 172.21.15.111:0/1254584707 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47300b3ca0 con 0x7f473006b390 2024-05-27T00:40:32.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f4734e5c640 1 --2- 172.21.15.111:0/1254584707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473006b390 0x7f47300cd740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f47240097a0 tx=0x7f472402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e89b481cb47c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f472ffff640 1 -- 172.21.15.111:0/1254584707 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724037070 con 0x7f473006b390 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f472ffff640 1 -- 172.21.15.111:0/1254584707 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4724032440 con 0x7f473006b390 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.517+0000 7f472ffff640 1 -- 172.21.15.111:0/1254584707 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724032810 con 0x7f473006b390 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 -- 172.21.15.111:0/1254584707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473006b390 msgr2=0x7f47300cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 --2- 172.21.15.111:0/1254584707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473006b390 0x7f47300cd740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f47240097a0 tx=0x7f472402a230 comp rx=0 tx=0).stop 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 -- 172.21.15.111:0/1254584707 shutdown_connections 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 --2- 172.21.15.111:0/1254584707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f473006b390 0x7f47300cd740 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 -- 172.21.15.111:0/1254584707 >> 172.21.15.111:0/1254584707 conn(0x7f47300b1c00 msgr2=0x7f47300b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 -- 172.21.15.111:0/1254584707 shutdown_connections 2024-05-27T00:40:32.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.518+0000 7f47378c1640 1 -- 172.21.15.111:0/1254584707 wait complete. 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.519+0000 7f47378c1640 1 Processor -- start 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.519+0000 7f47378c1640 1 -- start start 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f47378c1640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473006b390 0x7f4730159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f47378c1640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47300cdc80 0x7f47300ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f47378c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 0x7f47300ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f47378c1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4730072f60 con 0x7f4730144000 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f47378c1640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4730072de0 con 0x7f473006b390 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f47378c1640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f47300b4780 con 0x7f47300cdc80 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 0x7f47300ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 0x7f47300ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59402/0 (socket says 172.21.15.111:59402) 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 -- 172.21.15.111:0/2047085478 learned_addr learned my addr 172.21.15.111:0/2047085478 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:32.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 -- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47300cdc80 msgr2=0x7f47300ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47300cdc80 0x7f47300ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 -- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473006b390 msgr2=0x7f4730159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473006b390 0x7f4730159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.520+0000 7f4735e5e640 1 -- 172.21.15.111:0/2047085478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47301331f0 con 0x7f4730144000 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.521+0000 7f4735e5e640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 0x7f47300ae860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4728001eb0 tx=0x7f472800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.521+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728017070 con 0x7f4730144000 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.521+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47280052d0 con 0x7f4730144000 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.521+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47301334d0 con 0x7f4730144000 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.521+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728003810 con 0x7f4730144000 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.521+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4730133a90 con 0x7f4730144000 2024-05-27T00:40:32.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.522+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4730068680 con 0x7f4730144000 2024-05-27T00:40:32.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.522+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f472801a070 con 0x7f4730144000 2024-05-27T00:40:32.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.523+0000 7f472dffb640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f471c06be30 0x7f471c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:32.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.523+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f472808a320 con 0x7f4730144000 2024-05-27T00:40:32.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.523+0000 7f473565d640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f471c06be30 0x7f471c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:32.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.524+0000 7f473565d640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f471c06be30 0x7f471c06e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4720007fd0 tx=0x7f4720007d40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:32.528 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.527+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4728058bd0 con 0x7f4730144000 2024-05-27T00:40:32.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:32.737+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4730072130 con 0x7f4730144000 2024-05-27T00:40:33.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.397+0000 7f472dffb640 1 -- 172.21.15.111:0/2047085478 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f47280584d0 con 0x7f4730144000 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.399+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f471c06be30 msgr2=0x7f471c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.399+0000 7f47378c1640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f471c06be30 0x7f471c06e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4720007fd0 tx=0x7f4720007d40 comp rx=0 tx=0).stop 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.399+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 msgr2=0x7f47300ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.399+0000 7f47378c1640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 0x7f47300ae860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4728001eb0 tx=0x7f472800c890 comp rx=0 tx=0).stop 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 shutdown_connections 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f471c06be30 0x7f471c06e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4730144000 0x7f47300ae860 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f47300cdc80 0x7f47300ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 --2- 172.21.15.111:0/2047085478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473006b390 0x7f4730159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 >> 172.21.15.111:0/2047085478 conn(0x7f47300b1c00 msgr2=0x7f47300cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 shutdown_connections 2024-05-27T00:40:33.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.400+0000 7f47378c1640 1 -- 172.21.15.111:0/2047085478 wait complete. 2024-05-27T00:40:33.412 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-27T00:40:33.413 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:40:33.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.513+0000 7f730f209640 1 Processor -- start 2024-05-27T00:40:33.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.513+0000 7f730f209640 1 -- start start 2024-05-27T00:40:33.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730f209640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730f209640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7308162560 0x7f73080d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730f209640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f730806cc70 con 0x7f73080b77e0 2024-05-27T00:40:33.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730f209640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f73080d1e50 con 0x7f7308162560 2024-05-27T00:40:33.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730f209640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f73080d2130 con 0x7f73080b3630 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730d7a6640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7308162560 0x7f73080d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f7307fff640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f73080b77e0 0x7f7308161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58962/0 (socket says 172.21.15.111:58962) 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f7307fff640 1 -- 172.21.15.111:0/1342399065 learned_addr learned my addr 172.21.15.111:0/1342399065 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.514+0000 7f730d7a6640 1 --2- 172.21.15.111:0/1342399065 >> v2:172.21.15.111:3300/0 conn(0x7f7308162560 0x7f73080d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59414/0 (socket says 172.21.15.111:59414) 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.515+0000 7f730cfa5640 1 --2- 172.21.15.111:0/1342399065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.515+0000 7f730d7a6640 1 -- 172.21.15.111:0/1342399065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 msgr2=0x7f73080b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.515+0000 7f730d7a6640 1 --2- 172.21.15.111:0/1342399065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.515+0000 7f730d7a6640 1 -- 172.21.15.111:0/1342399065 >> v1:172.21.15.111:6789/0 conn(0x7f73080b77e0 legacy=0x7f7308161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.515+0000 7f730d7a6640 1 -- 172.21.15.111:0/1342399065 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73080d2410 con 0x7f7308162560 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.515+0000 7f730d7a6640 1 --2- 172.21.15.111:0/1342399065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308162560 0x7f73080d2c10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f73000097a0 tx=0x7f730002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493b7f6c9d68ac94 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f73077fe640 1 -- 172.21.15.111:0/1342399065 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300037070 con 0x7f7308162560 2024-05-27T00:40:33.517 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f73077fe640 1 -- 172.21.15.111:0/1342399065 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7300032440 con 0x7f7308162560 2024-05-27T00:40:33.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f73077fe640 1 -- 172.21.15.111:0/1342399065 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300032810 con 0x7f7308162560 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 -- 172.21.15.111:0/1342399065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308162560 msgr2=0x7f73080d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 --2- 172.21.15.111:0/1342399065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308162560 0x7f73080d2c10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f73000097a0 tx=0x7f730002a180 comp rx=0 tx=0).stop 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 -- 172.21.15.111:0/1342399065 shutdown_connections 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 --2- 172.21.15.111:0/1342399065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7308162560 0x7f73080d2c10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 --2- 172.21.15.111:0/1342399065 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 -- 172.21.15.111:0/1342399065 >> 172.21.15.111:0/1342399065 conn(0x7f73080d42c0 msgr2=0x7f73080d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 -- 172.21.15.111:0/1342399065 shutdown_connections 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.516+0000 7f730f209640 1 -- 172.21.15.111:0/1342399065 wait complete. 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.517+0000 7f730f209640 1 Processor -- start 2024-05-27T00:40:33.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.517+0000 7f730f209640 1 -- start start 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f730f209640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f730f209640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 0x7f7308160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f730f209640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7308162560 0x7f73080b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f730f209640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73080d32d0 con 0x7f73080b77e0 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f730f209640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f73080d3150 con 0x7f7308162560 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f730f209640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f73080d3450 con 0x7f73080b3630 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f7307fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 0x7f7308160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f7307fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 0x7f7308160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59430/0 (socket says 172.21.15.111:59430) 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.518+0000 7f7307fff640 1 -- 172.21.15.111:0/3772743064 learned_addr learned my addr 172.21.15.111:0/3772743064 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:33.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f730cfa5640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f730d7a6640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7308162560 0x7f73080b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f7307fff640 1 -- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 msgr2=0x7f73080b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f7307fff640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f7307fff640 1 -- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7308162560 msgr2=0x7f73080b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f7307fff640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7308162560 0x7f73080b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f7307fff640 1 -- 172.21.15.111:0/3772743064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73081f45f0 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f730cfa5640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f7307fff640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 0x7f7308160c40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f72f000c400 tx=0x7f72f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0017070 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f0005ce0 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00072f0 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73081f48d0 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.519+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73081f4e90 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.520+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73081f5380 con 0x7f73080b77e0 2024-05-27T00:40:33.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.520+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72f001a070 con 0x7f73080b77e0 2024-05-27T00:40:33.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.521+0000 7f73057fa640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.521+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f72f008a410 con 0x7f73080b77e0 2024-05-27T00:40:33.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.521+0000 7f730cfa5640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.522+0000 7f730cfa5640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f72e806be30 0x7f72e806e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009b20 tx=0x7f72f80039d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:33.526 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.525+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f72f0058c40 con 0x7f73080b77e0 2024-05-27T00:40:33.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.733+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7308068680 con 0x7f73080b77e0 2024-05-27T00:40:33.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.734+0000 7f73057fa640 1 -- 172.21.15.111:0/3772743064 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5190 (secure 0 0 0) 0x7f72f0058540 con 0x7f73080b77e0 2024-05-27T00:40:33.735 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:33.735 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1589050951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1589050951},{"type":"v1","addr":"172.21.15.153:6836","nonce":1589050951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/662515803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":662515803},{"type":"v1","addr":"172.21.15.153:6837","nonce":662515803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1197288581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1197288581},{"type":"v1","addr":"172.21.15.111:6834","nonce":1197288581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:40:30.176062+0000","modified":"2024-05-27T00:40:32.738961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/3118800442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3118800442},{"type":"v1","addr":"172.21.15.111:6835","nonce":3118800442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5418,"qdb_cluster":[5418]},"id":2}]} 2024-05-27T00:40:33.735 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 17 2024-05-27T00:40:33.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f72e806be30 msgr2=0x7f72e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f72e806be30 0x7f72e806e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009b20 tx=0x7f72f80039d0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 msgr2=0x7f7308160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.737 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 0x7f7308160c40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f72f000c400 tx=0x7f72f000c8c0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 shutdown_connections 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7308162560 0x7f73080b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73080b77e0 0x7f7308160c40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 --2- 172.21.15.111:0/3772743064 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f73080b3630 0x7f73080b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 >> 172.21.15.111:0/3772743064 conn(0x7f73080d42c0 msgr2=0x7f73080b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 shutdown_connections 2024-05-27T00:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.736+0000 7f730f209640 1 -- 172.21.15.111:0/3772743064 wait complete. 2024-05-27T00:40:33.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T00:40:33.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.848+0000 7f849969a640 1 Processor -- start 2024-05-27T00:40:33.849 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.848+0000 7f849969a640 1 -- start start 2024-05-27T00:40:33.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f849969a640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f84940ce140 0x7f84940cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f849969a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8494161ec0 0x7f8494162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.850 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f849969a640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84940b5460 con 0x7f84940b4fd0 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f849969a640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f84941628b0 con 0x7f84940ce140 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f849969a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f849415f7c0 con 0x7f8494161ec0 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f84940ce140 0x7f84940cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f84937fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8494161ec0 0x7f8494162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f84940ce140 0x7f84940cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:50112/0 (socket says 172.21.15.111:50112) 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 -- 172.21.15.111:0/3528719193 learned_addr learned my addr 172.21.15.111:0/3528719193 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f84927fc640 1 --1- 172.21.15.111:0/3528719193 >> v1:172.21.15.153:6789/0 conn(0x7f84940b4fd0 0x7f849415f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:41648/0 (socket says 172.21.15.111:41648) 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 -- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8494161ec0 msgr2=0x7f8494162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 --2- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8494161ec0 0x7f8494162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 -- 172.21.15.111:0/3528719193 >> v1:172.21.15.153:6789/0 conn(0x7f84940b4fd0 legacy=0x7f849415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f8492ffd640 1 -- 172.21.15.111:0/3528719193 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84940d0200 con 0x7f84940ce140 2024-05-27T00:40:33.851 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.849+0000 7f84937fe640 1 --2- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8494161ec0 0x7f8494162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.850+0000 7f8492ffd640 1 --2- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940ce140 0x7f84940cfb20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8484009870 tx=0x7f848402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e6d1abb003a8db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.850+0000 7f8491ffb640 1 -- 172.21.15.111:0/3528719193 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484037070 con 0x7f84940ce140 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.850+0000 7f8491ffb640 1 -- 172.21.15.111:0/3528719193 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8484032440 con 0x7f84940ce140 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.850+0000 7f8491ffb640 1 -- 172.21.15.111:0/3528719193 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484032810 con 0x7f84940ce140 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 -- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940ce140 msgr2=0x7f84940cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 --2- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940ce140 0x7f84940cfb20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8484009870 tx=0x7f848402a130 comp rx=0 tx=0).stop 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 -- 172.21.15.111:0/3528719193 shutdown_connections 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 --2- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8494161ec0 0x7f8494162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 --2- 172.21.15.111:0/3528719193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940ce140 0x7f84940cfb20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 -- 172.21.15.111:0/3528719193 >> 172.21.15.111:0/3528719193 conn(0x7f849406ba70 msgr2=0x7f84940d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:33.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 -- 172.21.15.111:0/3528719193 shutdown_connections 2024-05-27T00:40:33.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.851+0000 7f849969a640 1 -- 172.21.15.111:0/3528719193 wait complete. 2024-05-27T00:40:33.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.852+0000 7f849969a640 1 Processor -- start 2024-05-27T00:40:33.854 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.852+0000 7f849969a640 1 -- start start 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f849969a640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940b4fd0 0x7f84941524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f849969a640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84940ce140 0x7f84940c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f849969a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 0x7f84940c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f849969a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f849415e8a0 con 0x7f8494161ec0 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f849969a640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f84940d07c0 con 0x7f84940b4fd0 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f849969a640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f84940ce520 con 0x7f84940ce140 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 0x7f84940c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 0x7f84940c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59446/0 (socket says 172.21.15.111:59446) 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 -- 172.21.15.111:0/1589441446 learned_addr learned my addr 172.21.15.111:0/1589441446 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f8492ffd640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940b4fd0 0x7f84941524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84927fc640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84940ce140 0x7f84940c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 -- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84940ce140 msgr2=0x7f84940c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84940ce140 0x7f84940c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 -- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940b4fd0 msgr2=0x7f84941524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940b4fd0 0x7f84941524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f84937fe640 1 -- 172.21.15.111:0/1589441446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84940c76c0 con 0x7f8494161ec0 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.853+0000 7f8492ffd640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940b4fd0 0x7f84941524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.854+0000 7f84937fe640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 0x7f84940c82c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f848c00efc0 tx=0x7f848c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.854+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c018030 con 0x7f8494161ec0 2024-05-27T00:40:33.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.854+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c00cdf0 con 0x7f8494161ec0 2024-05-27T00:40:33.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.854+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c003bb0 con 0x7f8494161ec0 2024-05-27T00:40:33.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.854+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8494152a30 con 0x7f8494161ec0 2024-05-27T00:40:33.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.854+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84940c6900 con 0x7f8494161ec0 2024-05-27T00:40:33.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.855+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8494068680 con 0x7f8494161ec0 2024-05-27T00:40:33.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.855+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f848c005d10 con 0x7f8494161ec0 2024-05-27T00:40:33.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.856+0000 7f847b7fe640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f848006be30 0x7f848006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:33.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.856+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f848c08b990 con 0x7f8494161ec0 2024-05-27T00:40:33.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.856+0000 7f8492ffd640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f848006be30 0x7f848006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:33.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.857+0000 7f8492ffd640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f848006be30 0x7f848006e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8484034000 tx=0x7f848402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:33.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:33.860+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f848c022070 con 0x7f8494161ec0 2024-05-27T00:40:34.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.036+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84940d13e0 con 0x7f8494161ec0 2024-05-27T00:40:34.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.037+0000 7f847b7fe640 1 -- 172.21.15.111:0/1589441446 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16990 (secure 0 0 0) 0x7f848c010040 con 0x7f8494161ec0 2024-05-27T00:40:34.038 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:34.038 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":39,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T00:40:31.839112+0000","last_up_change":"2024-05-27T00:39:07.975018+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:40:28.415554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-27T00:40:29.147789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists","up"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f848006be30 msgr2=0x7f848006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f848006be30 0x7f848006e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8484034000 tx=0x7f848402e000 comp rx=0 tx=0).stop 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 msgr2=0x7f84940c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 0x7f84940c82c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f848c00efc0 tx=0x7f848c00c490 comp rx=0 tx=0).stop 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 shutdown_connections 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f848006be30 0x7f848006e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8494161ec0 0x7f84940c82c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f84940ce140 0x7f84940c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 --2- 172.21.15.111:0/1589441446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84940b4fd0 0x7f84941524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 >> 172.21.15.111:0/1589441446 conn(0x7f849406ba70 msgr2=0x7f849415efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:34.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 shutdown_connections 2024-05-27T00:40:34.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.039+0000 7f849969a640 1 -- 172.21.15.111:0/1589441446 wait complete. 2024-05-27T00:40:34.051 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-27T00:40:34.052 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:40:34.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.151+0000 7f8a072d4640 1 Processor -- start 2024-05-27T00:40:34.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.151+0000 7f8a072d4640 1 -- start start 2024-05-27T00:40:34.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a072d4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a00144000 0x7f8a000b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a072d4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a0006b390 0x7f8a000cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a072d4640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000b3440 con 0x7f8a000cdc80 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a072d4640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f8a000b42f0 con 0x7f8a0006b390 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a072d4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000b36e0 con 0x7f8a00144000 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a0006b390 0x7f8a000cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8a0006b390 0x7f8a000cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59460/0 (socket says 172.21.15.111:59460) 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 -- 172.21.15.111:0/2662121575 learned_addr learned my addr 172.21.15.111:0/2662121575 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 -- 172.21.15.111:0/2662121575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a00144000 msgr2=0x7f8a000b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 --2- 172.21.15.111:0/2662121575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a00144000 0x7f8a000b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 -- 172.21.15.111:0/2662121575 >> v1:172.21.15.153:6789/0 conn(0x7f8a000cdc80 legacy=0x7f8a000726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.152+0000 7f8a0486f640 1 -- 172.21.15.111:0/2662121575 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000b39c0 con 0x7f8a0006b390 2024-05-27T00:40:34.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.153+0000 7f8a0486f640 1 --2- 172.21.15.111:0/2662121575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0006b390 0x7f8a000cd740 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566f1d3cd81609e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.153+0000 7f89f7fff640 1 -- 172.21.15.111:0/2662121575 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0037070 con 0x7f8a0006b390 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.153+0000 7f89f7fff640 1 -- 172.21.15.111:0/2662121575 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f0032440 con 0x7f8a0006b390 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.153+0000 7f89f7fff640 1 -- 172.21.15.111:0/2662121575 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0032810 con 0x7f8a0006b390 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 -- 172.21.15.111:0/2662121575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0006b390 msgr2=0x7f8a000cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2662121575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0006b390 0x7f8a000cd740 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f002a210 comp rx=0 tx=0).stop 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 -- 172.21.15.111:0/2662121575 shutdown_connections 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2662121575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0006b390 0x7f8a000cd740 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2662121575 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a00144000 0x7f8a000b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.156 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 -- 172.21.15.111:0/2662121575 >> 172.21.15.111:0/2662121575 conn(0x7f8a000b1c00 msgr2=0x7f8a000b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:34.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 -- 172.21.15.111:0/2662121575 shutdown_connections 2024-05-27T00:40:34.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.154+0000 7f8a072d4640 1 -- 172.21.15.111:0/2662121575 wait complete. 2024-05-27T00:40:34.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.155+0000 7f8a072d4640 1 Processor -- start 2024-05-27T00:40:34.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.155+0000 7f8a072d4640 1 -- start start 2024-05-27T00:40:34.159 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a0006b390 0x7f8a000ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 0x7f8a0013d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a00144000 0x7f8a0006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00072de0 con 0x7f8a000cdc80 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b3f80 con 0x7f8a00144000 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f8a00072f60 con 0x7f8a0006b390 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 0x7f8a0013d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 0x7f8a0013d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59466/0 (socket says 172.21.15.111:59466) 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 -- 172.21.15.111:0/2337677200 learned_addr learned my addr 172.21.15.111:0/2337677200 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a05871640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a00144000 0x7f8a0006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a05070640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a0006b390 0x7f8a000ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 -- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a0006b390 msgr2=0x7f8a000ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a0006b390 0x7f8a000ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 -- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a00144000 msgr2=0x7f8a0006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a00144000 0x7f8a0006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 -- 172.21.15.111:0/2337677200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001e13a0 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a05871640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a00144000 0x7f8a0006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a0486f640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 0x7f8a0013d700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0041070 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f003c870 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001e0de0 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.156+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0037040 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.157+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0014fa00 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.158+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00068680 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.158+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89f0044070 con 0x7f8a000cdc80 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.158+0000 7f89f5ffb640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89e406be30 0x7f89e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.159+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f89f00abaa0 con 0x7f8a000cdc80 2024-05-27T00:40:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.159+0000 7f8a05070640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89e406be30 0x7f89e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.159+0000 7f8a05070640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89e406be30 0x7f89e406e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f89fc007dc0 tx=0x7f89fc0072f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.163+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f89f007a350 con 0x7f8a000cdc80 2024-05-27T00:40:34.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.332+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a000afb30 con 0x7f89e406be30 2024-05-27T00:40:34.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.338+0000 7f89f5ffb640 1 -- 172.21.15.111:0/2337677200 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240038 (secure 0 0 0) 0x7f8a000afb30 con 0x7f89e406be30 2024-05-27T00:40:34.342 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:34.343 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:40:34.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89e406be30 msgr2=0x7f89e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89e406be30 0x7f89e406e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f89fc007dc0 tx=0x7f89fc0072f0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 msgr2=0x7f8a0013d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 0x7f8a0013d700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f00046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 shutdown_connections 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f89e406be30 0x7f89e406e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a00144000 0x7f8a0006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a000cdc80 0x7f8a0013d700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 --2- 172.21.15.111:0/2337677200 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f8a0006b390 0x7f8a000ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 >> 172.21.15.111:0/2337677200 conn(0x7f8a000b1c00 msgr2=0x7f8a000cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 shutdown_connections 2024-05-27T00:40:34.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.341+0000 7f8a072d4640 1 -- 172.21.15.111:0/2337677200 wait complete. 2024-05-27T00:40:34.353 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-05-27T00:40:32.942472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220504,"kb_used_data":4308,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754216,"statfs":{"total":773094113280,"available":772868317184,"internally_reserved":0,"allocated":4411392,"data_stored":2551681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.120300"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826460+0000","last_change":"2024-05-27T00:40:28.826460+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826460+0000","last_undegraded":"2024-05-27T00:40:28.826460+0000","last_fullsized":"2024-05-27T00:40:28.826460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822423+0000","last_change":"2024-05-27T00:40:28.822423+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822423+0000","last_undegraded":"2024-05-27T00:40:28.822423+0000","last_fullsized":"2024-05-27T00:40:28.822423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824374+0000","last_change":"2024-05-27T00:40:28.824374+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824374+0000","last_undegraded":"2024-05-27T00:40:28.824374+0000","last_fullsized":"2024-05-27T00:40:28.824374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.823656+0000","last_change":"2024-05-27T00:40:28.823656+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.823656+0000","last_undegraded":"2024-05-27T00:40:28.823656+0000","last_fullsized":"2024-05-27T00:40:28.823656+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825293+0000","last_change":"2024-05-27T00:40:28.825293+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825293+0000","last_undegraded":"2024-05-27T00:40:28.825293+0000","last_fullsized":"2024-05-27T00:40:28.825293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825816+0000","last_change":"2024-05-27T00:40:28.825816+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825816+0000","last_undegraded":"2024-05-27T00:40:28.825816+0000","last_fullsized":"2024-05-27T00:40:28.825816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824366+0000","last_change":"2024-05-27T00:40:28.824366+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824366+0000","last_undegraded":"2024-05-27T00:40:28.824366+0000","last_fullsized":"2024-05-27T00:40:28.824366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.823620+0000","last_change":"2024-05-27T00:40:28.823620+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.823620+0000","last_undegraded":"2024-05-27T00:40:28.823620+0000","last_fullsized":"2024-05-27T00:40:28.823620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825346+0000","last_change":"2024-05-27T00:40:28.825346+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825346+0000","last_undegraded":"2024-05-27T00:40:28.825346+0000","last_fullsized":"2024-05-27T00:40:28.825346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825324+0000","last_change":"2024-05-27T00:40:28.825324+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825324+0000","last_undegraded":"2024-05-27T00:40:28.825324+0000","last_fullsized":"2024-05-27T00:40:28.825324+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825708+0000","last_change":"2024-05-27T00:40:28.825708+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825708+0000","last_undegraded":"2024-05-27T00:40:28.825708+0000","last_fullsized":"2024-05-27T00:40:28.825708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825124+0000","last_change":"2024-05-27T00:40:28.825124+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825124+0000","last_undegraded":"2024-05-27T00:40:28.825124+0000","last_fullsized":"2024-05-27T00:40:28.825124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825130+0000","last_change":"2024-05-27T00:40:28.825130+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825130+0000","last_undegraded":"2024-05-27T00:40:28.825130+0000","last_fullsized":"2024-05-27T00:40:28.825130+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824397+0000","last_change":"2024-05-27T00:40:28.824397+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824397+0000","last_undegraded":"2024-05-27T00:40:28.824397+0000","last_fullsized":"2024-05-27T00:40:28.824397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825966+0000","last_change":"2024-05-27T00:40:28.825966+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825966+0000","last_undegraded":"2024-05-27T00:40:28.825966+0000","last_fullsized":"2024-05-27T00:40:28.825966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826578+0000","last_change":"2024-05-27T00:40:28.826578+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826578+0000","last_undegraded":"2024-05-27T00:40:28.826578+0000","last_fullsized":"2024-05-27T00:40:28.826578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825955+0000","last_change":"2024-05-27T00:40:28.825955+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825955+0000","last_undegraded":"2024-05-27T00:40:28.825955+0000","last_fullsized":"2024-05-27T00:40:28.825955+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.823294+0000","last_change":"2024-05-27T00:40:28.823294+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.823294+0000","last_undegraded":"2024-05-27T00:40:28.823294+0000","last_fullsized":"2024-05-27T00:40:28.823294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820877+0000","last_change":"2024-05-27T00:40:28.820877+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820877+0000","last_undegraded":"2024-05-27T00:40:28.820877+0000","last_fullsized":"2024-05-27T00:40:28.820877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824365+0000","last_change":"2024-05-27T00:40:28.824365+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824365+0000","last_undegraded":"2024-05-27T00:40:28.824365+0000","last_fullsized":"2024-05-27T00:40:28.824365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820547+0000","last_change":"2024-05-27T00:40:28.820547+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820547+0000","last_undegraded":"2024-05-27T00:40:28.820547+0000","last_fullsized":"2024-05-27T00:40:28.820547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820541+0000","last_change":"2024-05-27T00:40:28.820541+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820541+0000","last_undegraded":"2024-05-27T00:40:28.820541+0000","last_fullsized":"2024-05-27T00:40:28.820541+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826314+0000","last_change":"2024-05-27T00:40:28.826314+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826314+0000","last_undegraded":"2024-05-27T00:40:28.826314+0000","last_fullsized":"2024-05-27T00:40:28.826314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820528+0000","last_change":"2024-05-27T00:40:28.820528+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820528+0000","last_undegraded":"2024-05-27T00:40:28.820528+0000","last_fullsized":"2024-05-27T00:40:28.820528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825358+0000","last_change":"2024-05-27T00:40:28.825358+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825358+0000","last_undegraded":"2024-05-27T00:40:28.825358+0000","last_fullsized":"2024-05-27T00:40:28.825358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820531+0000","last_change":"2024-05-27T00:40:28.820531+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820531+0000","last_undegraded":"2024-05-27T00:40:28.820531+0000","last_fullsized":"2024-05-27T00:40:28.820531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822450+0000","last_change":"2024-05-27T00:40:28.822450+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822450+0000","last_undegraded":"2024-05-27T00:40:28.822450+0000","last_fullsized":"2024-05-27T00:40:28.822450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822470+0000","last_change":"2024-05-27T00:40:28.822470+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822470+0000","last_undegraded":"2024-05-27T00:40:28.822470+0000","last_fullsized":"2024-05-27T00:40:28.822470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826370+0000","last_change":"2024-05-27T00:40:28.826370+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826370+0000","last_undegraded":"2024-05-27T00:40:28.826370+0000","last_fullsized":"2024-05-27T00:40:28.826370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822950+0000","last_change":"2024-05-27T00:40:28.822950+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822950+0000","last_undegraded":"2024-05-27T00:40:28.822950+0000","last_fullsized":"2024-05-27T00:40:28.822950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826139+0000","last_change":"2024-05-27T00:40:28.826139+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826139+0000","last_undegraded":"2024-05-27T00:40:28.826139+0000","last_fullsized":"2024-05-27T00:40:28.826139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.827928+0000","last_change":"2024-05-27T00:40:28.827928+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.827928+0000","last_undegraded":"2024-05-27T00:40:28.827928+0000","last_fullsized":"2024-05-27T00:40:28.827928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-27T00:40:28.826805+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:40:28.826805+0000","last_peered":"2024-05-27T00:40:28.826805+0000","last_clean":"2024-05-27T00:40:28.826805+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:40:28.826805+0000","last_undegraded":"2024-05-27T00:40:28.826805+0000","last_fullsized":"2024-05-27T00:40:28.826805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":760960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:40:34.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:40:34.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.493+0000 7f9cf4238640 1 Processor -- start 2024-05-27T00:40:34.494 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.493+0000 7f9cf4238640 1 -- start start 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf4238640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9cec13ef60 0x7f9cec0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf4238640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9cec13f340 0x7f9cec0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf4238640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0cce00 0x7f9cec141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf4238640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f9cec0cf210 con 0x7f9cec13f340 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf4238640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f9cec142650 con 0x7f9cec13ef60 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf4238640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9cec0cebc0 con 0x7f9cec0cce00 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf17d3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9cec13f340 0x7f9cec0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf17d3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9cec13f340 0x7f9cec0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59478/0 (socket says 172.21.15.111:59478) 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.494+0000 7f9cf17d3640 1 -- 172.21.15.111:0/4168493388 learned_addr learned my addr 172.21.15.111:0/4168493388 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf17d3640 1 -- 172.21.15.111:0/4168493388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0cce00 msgr2=0x7f9cec141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4168493388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0cce00 0x7f9cec141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf17d3640 1 -- 172.21.15.111:0/4168493388 >> v2:172.21.15.153:3300/0 conn(0x7f9cec13ef60 msgr2=0x7f9cec0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:34.496 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4168493388 >> v2:172.21.15.153:3300/0 conn(0x7f9cec13ef60 0x7f9cec0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.497 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf17d3640 1 -- 172.21.15.111:0/4168493388 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec0ceea0 con 0x7f9cec13f340 2024-05-27T00:40:34.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4168493388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec13f340 0x7f9cec0cc8c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0742a0 tx=0x7f9ce002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33326fb99bc0efef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf0fd2640 1 -- 172.21.15.111:0/4168493388 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0037070 con 0x7f9cec13f340 2024-05-27T00:40:34.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.495+0000 7f9cf0fd2640 1 -- 172.21.15.111:0/4168493388 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0032440 con 0x7f9cec13f340 2024-05-27T00:40:34.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf0fd2640 1 -- 172.21.15.111:0/4168493388 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0032810 con 0x7f9cec13f340 2024-05-27T00:40:34.498 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 -- 172.21.15.111:0/4168493388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec13f340 msgr2=0x7f9cec0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4168493388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec13f340 0x7f9cec0cc8c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0742a0 tx=0x7f9ce002a1e0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 -- 172.21.15.111:0/4168493388 shutdown_connections 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4168493388 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec0cce00 0x7f9cec141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4168493388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec13f340 0x7f9cec0cc8c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4168493388 >> v2:172.21.15.153:3300/0 conn(0x7f9cec13ef60 0x7f9cec0cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 -- 172.21.15.111:0/4168493388 >> 172.21.15.111:0/4168493388 conn(0x7f9cec0b1c00 msgr2=0x7f9cec0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 -- 172.21.15.111:0/4168493388 shutdown_connections 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.496+0000 7f9cf4238640 1 -- 172.21.15.111:0/4168493388 wait complete. 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.497+0000 7f9cf4238640 1 Processor -- start 2024-05-27T00:40:34.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.497+0000 7f9cf4238640 1 -- start start 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf4238640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 0x7f9cec15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf4238640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec13ef60 0x7f9cec15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf4238640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec13f340 0x7f9cec15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf4238640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec141e60 con 0x7f9cec0cce00 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf4238640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec0ce890 con 0x7f9cec13ef60 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf4238640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f9cec141fe0 con 0x7f9cec13f340 2024-05-27T00:40:34.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 0x7f9cec15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 0x7f9cec15be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59482/0 (socket says 172.21.15.111:59482) 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 -- 172.21.15.111:0/4233501655 learned_addr learned my addr 172.21.15.111:0/4233501655 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf27d5640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec13f340 0x7f9cec15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec13ef60 0x7f9cec15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 -- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec13f340 msgr2=0x7f9cec15a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec13f340 0x7f9cec15a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 -- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec13ef60 msgr2=0x7f9cec15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec13ef60 0x7f9cec15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.498+0000 7f9cf1fd4640 1 -- 172.21.15.111:0/4233501655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec15afc0 con 0x7f9cec0cce00 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec13ef60 0x7f9cec15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cf27d5640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec13f340 0x7f9cec15a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cf1fd4640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 0x7f9cec15be90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00ba50 tx=0x7f9cdc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc019070 con 0x7f9cec0cce00 2024-05-27T00:40:34.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cdc00cc00 con 0x7f9cec0cce00 2024-05-27T00:40:34.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc014440 con 0x7f9cec0cce00 2024-05-27T00:40:34.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cec15b2a0 con 0x7f9cec0cce00 2024-05-27T00:40:34.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.499+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cec15b860 con 0x7f9cec0cce00 2024-05-27T00:40:34.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.500+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cec0cdeb0 con 0x7f9cec0cce00 2024-05-27T00:40:34.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.500+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cdc01c070 con 0x7f9cec0cce00 2024-05-27T00:40:34.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.501+0000 7f9cea7fc640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9cd806be60 0x7f9cd806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.501+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9cdc08ae00 con 0x7f9cec0cce00 2024-05-27T00:40:34.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.501+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9cd806be60 0x7f9cd806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.502+0000 7f9cf17d3640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9cd806be60 0x7f9cd806e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0034000 tx=0x7f9ce002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.505+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9cec0cdeb0 con 0x7f9cec0cce00 2024-05-27T00:40:34.675 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.675+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9cec068680 con 0x7f9cd806be60 2024-05-27T00:40:34.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.681+0000 7f9cea7fc640 1 -- 172.21.15.111:0/4233501655 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240038 (secure 0 0 0) 0x7f9cec068680 con 0x7f9cd806be60 2024-05-27T00:40:34.684 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:34.685 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:40:34.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.683+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9cd806be60 msgr2=0x7f9cd806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.683+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9cd806be60 0x7f9cd806e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0034000 tx=0x7f9ce002e000 comp rx=0 tx=0).stop 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.683+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 msgr2=0x7f9cec15be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.683+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 0x7f9cec15be90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00ba50 tx=0x7f9cdc00bf10 comp rx=0 tx=0).stop 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.683+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 shutdown_connections 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.683+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f9cd806be60 0x7f9cd806e320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.684+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f9cec13f340 0x7f9cec15a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.684+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cec13ef60 0x7f9cec15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.684+0000 7f9cf4238640 1 --2- 172.21.15.111:0/4233501655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cec0cce00 0x7f9cec15be90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.684+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 >> 172.21.15.111:0/4233501655 conn(0x7f9cec0b1c00 msgr2=0x7f9cec15ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.684+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 shutdown_connections 2024-05-27T00:40:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.684+0000 7f9cf4238640 1 -- 172.21.15.111:0/4233501655 wait complete. 2024-05-27T00:40:34.695 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-05-27T00:40:32.942472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220504,"kb_used_data":4308,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754216,"statfs":{"total":773094113280,"available":772868317184,"internally_reserved":0,"allocated":4411392,"data_stored":2551681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.120300"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826460+0000","last_change":"2024-05-27T00:40:28.826460+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826460+0000","last_undegraded":"2024-05-27T00:40:28.826460+0000","last_fullsized":"2024-05-27T00:40:28.826460+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822423+0000","last_change":"2024-05-27T00:40:28.822423+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822423+0000","last_undegraded":"2024-05-27T00:40:28.822423+0000","last_fullsized":"2024-05-27T00:40:28.822423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824374+0000","last_change":"2024-05-27T00:40:28.824374+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824374+0000","last_undegraded":"2024-05-27T00:40:28.824374+0000","last_fullsized":"2024-05-27T00:40:28.824374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.823656+0000","last_change":"2024-05-27T00:40:28.823656+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.823656+0000","last_undegraded":"2024-05-27T00:40:28.823656+0000","last_fullsized":"2024-05-27T00:40:28.823656+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825293+0000","last_change":"2024-05-27T00:40:28.825293+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825293+0000","last_undegraded":"2024-05-27T00:40:28.825293+0000","last_fullsized":"2024-05-27T00:40:28.825293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825816+0000","last_change":"2024-05-27T00:40:28.825816+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825816+0000","last_undegraded":"2024-05-27T00:40:28.825816+0000","last_fullsized":"2024-05-27T00:40:28.825816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824366+0000","last_change":"2024-05-27T00:40:28.824366+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824366+0000","last_undegraded":"2024-05-27T00:40:28.824366+0000","last_fullsized":"2024-05-27T00:40:28.824366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.823620+0000","last_change":"2024-05-27T00:40:28.823620+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.823620+0000","last_undegraded":"2024-05-27T00:40:28.823620+0000","last_fullsized":"2024-05-27T00:40:28.823620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825346+0000","last_change":"2024-05-27T00:40:28.825346+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825346+0000","last_undegraded":"2024-05-27T00:40:28.825346+0000","last_fullsized":"2024-05-27T00:40:28.825346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825324+0000","last_change":"2024-05-27T00:40:28.825324+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825324+0000","last_undegraded":"2024-05-27T00:40:28.825324+0000","last_fullsized":"2024-05-27T00:40:28.825324+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825708+0000","last_change":"2024-05-27T00:40:28.825708+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825708+0000","last_undegraded":"2024-05-27T00:40:28.825708+0000","last_fullsized":"2024-05-27T00:40:28.825708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825124+0000","last_change":"2024-05-27T00:40:28.825124+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825124+0000","last_undegraded":"2024-05-27T00:40:28.825124+0000","last_fullsized":"2024-05-27T00:40:28.825124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825130+0000","last_change":"2024-05-27T00:40:28.825130+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825130+0000","last_undegraded":"2024-05-27T00:40:28.825130+0000","last_fullsized":"2024-05-27T00:40:28.825130+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824397+0000","last_change":"2024-05-27T00:40:28.824397+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824397+0000","last_undegraded":"2024-05-27T00:40:28.824397+0000","last_fullsized":"2024-05-27T00:40:28.824397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825966+0000","last_change":"2024-05-27T00:40:28.825966+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825966+0000","last_undegraded":"2024-05-27T00:40:28.825966+0000","last_fullsized":"2024-05-27T00:40:28.825966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826578+0000","last_change":"2024-05-27T00:40:28.826578+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826578+0000","last_undegraded":"2024-05-27T00:40:28.826578+0000","last_fullsized":"2024-05-27T00:40:28.826578+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825955+0000","last_change":"2024-05-27T00:40:28.825955+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825955+0000","last_undegraded":"2024-05-27T00:40:28.825955+0000","last_fullsized":"2024-05-27T00:40:28.825955+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.823294+0000","last_change":"2024-05-27T00:40:28.823294+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.823294+0000","last_undegraded":"2024-05-27T00:40:28.823294+0000","last_fullsized":"2024-05-27T00:40:28.823294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820877+0000","last_change":"2024-05-27T00:40:28.820877+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820877+0000","last_undegraded":"2024-05-27T00:40:28.820877+0000","last_fullsized":"2024-05-27T00:40:28.820877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.824365+0000","last_change":"2024-05-27T00:40:28.824365+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.824365+0000","last_undegraded":"2024-05-27T00:40:28.824365+0000","last_fullsized":"2024-05-27T00:40:28.824365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820547+0000","last_change":"2024-05-27T00:40:28.820547+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820547+0000","last_undegraded":"2024-05-27T00:40:28.820547+0000","last_fullsized":"2024-05-27T00:40:28.820547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820541+0000","last_change":"2024-05-27T00:40:28.820541+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820541+0000","last_undegraded":"2024-05-27T00:40:28.820541+0000","last_fullsized":"2024-05-27T00:40:28.820541+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826314+0000","last_change":"2024-05-27T00:40:28.826314+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826314+0000","last_undegraded":"2024-05-27T00:40:28.826314+0000","last_fullsized":"2024-05-27T00:40:28.826314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820528+0000","last_change":"2024-05-27T00:40:28.820528+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820528+0000","last_undegraded":"2024-05-27T00:40:28.820528+0000","last_fullsized":"2024-05-27T00:40:28.820528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.825358+0000","last_change":"2024-05-27T00:40:28.825358+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.825358+0000","last_undegraded":"2024-05-27T00:40:28.825358+0000","last_fullsized":"2024-05-27T00:40:28.825358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.820531+0000","last_change":"2024-05-27T00:40:28.820531+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.820531+0000","last_undegraded":"2024-05-27T00:40:28.820531+0000","last_fullsized":"2024-05-27T00:40:28.820531+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822450+0000","last_change":"2024-05-27T00:40:28.822450+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822450+0000","last_undegraded":"2024-05-27T00:40:28.822450+0000","last_fullsized":"2024-05-27T00:40:28.822450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822470+0000","last_change":"2024-05-27T00:40:28.822470+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822470+0000","last_undegraded":"2024-05-27T00:40:28.822470+0000","last_fullsized":"2024-05-27T00:40:28.822470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826370+0000","last_change":"2024-05-27T00:40:28.826370+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826370+0000","last_undegraded":"2024-05-27T00:40:28.826370+0000","last_fullsized":"2024-05-27T00:40:28.826370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.822950+0000","last_change":"2024-05-27T00:40:28.822950+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.822950+0000","last_undegraded":"2024-05-27T00:40:28.822950+0000","last_fullsized":"2024-05-27T00:40:28.822950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.826139+0000","last_change":"2024-05-27T00:40:28.826139+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.826139+0000","last_undegraded":"2024-05-27T00:40:28.826139+0000","last_fullsized":"2024-05-27T00:40:28.826139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-27T00:40:29.817168+0000","last_change":"2024-05-27T00:40:29.817168+0000","last_active":"2024-05-27T00:40:29.817168+0000","last_peered":"2024-05-27T00:40:29.817168+0000","last_clean":"2024-05-27T00:40:29.817168+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:29.817168+0000","last_undegraded":"2024-05-27T00:40:29.817168+0000","last_fullsized":"2024-05-27T00:40:29.817168+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-05-27T00:40:28.827928+0000","last_change":"2024-05-27T00:40:28.827928+0000","last_active":"2024-05-27T00:40:28.815391+0000","last_peered":"2024-05-27T00:40:28.815391+0000","last_clean":"2024-05-27T00:40:28.815391+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-27T00:40:28.827928+0000","last_undegraded":"2024-05-27T00:40:28.827928+0000","last_fullsized":"2024-05-27T00:40:28.827928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-27T00:40:28.826805+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:40:28.826805+0000","last_peered":"2024-05-27T00:40:28.826805+0000","last_clean":"2024-05-27T00:40:28.826805+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:40:28.826805+0000","last_undegraded":"2024-05-27T00:40:28.826805+0000","last_fullsized":"2024-05-27T00:40:28.826805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":760960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":767169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:40:34.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:40:34.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.835+0000 7fb568370640 1 Processor -- start 2024-05-27T00:40:34.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.835+0000 7fb568370640 1 -- start start 2024-05-27T00:40:34.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb568370640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb5600b5e40 0x7fb56006f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb568370640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb560145350 0x7fb56006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb568370640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb560145af0 con 0x7fb560144cd0 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb568370640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb560160500 con 0x7fb560145350 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb568370640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb56015fbd0 con 0x7fb5600b5e40 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56690d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb560145350 0x7fb56006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56610c640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb560144cd0 0x7fb5600b5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58978/0 (socket says 172.21.15.111:58978) 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56610c640 1 -- 172.21.15.111:0/4232173366 learned_addr learned my addr 172.21.15.111:0/4232173366 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56690d640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb560145350 0x7fb56006c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59484/0 (socket says 172.21.15.111:59484) 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56590b640 1 --2- 172.21.15.111:0/4232173366 >> v2:172.21.15.153:3300/0 conn(0x7fb5600b5e40 0x7fb56006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56690d640 1 -- 172.21.15.111:0/4232173366 >> v2:172.21.15.153:3300/0 conn(0x7fb5600b5e40 msgr2=0x7fb56006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56690d640 1 --2- 172.21.15.111:0/4232173366 >> v2:172.21.15.153:3300/0 conn(0x7fb5600b5e40 0x7fb56006f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56690d640 1 -- 172.21.15.111:0/4232173366 >> v1:172.21.15.111:6789/0 conn(0x7fb560144cd0 legacy=0x7fb5600b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56690d640 1 -- 172.21.15.111:0/4232173366 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56015feb0 con 0x7fb560145350 2024-05-27T00:40:34.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.836+0000 7fb56510a640 1 -- 172.21.15.111:0/4232173366 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 73165667 0 0) 0x7fb560145af0 con 0x7fb560144cd0 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.837+0000 7fb56690d640 1 --2- 172.21.15.111:0/4232173366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560145350 0x7fb56006c830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb56015fb30 tx=0x7fb55802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f67111f318f2801 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.837+0000 7fb56510a640 1 -- 172.21.15.111:0/4232173366 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558007740 con 0x7fb560145350 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.837+0000 7fb56510a640 1 -- 172.21.15.111:0/4232173366 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5580078e0 con 0x7fb560145350 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.837+0000 7fb56510a640 1 -- 172.21.15.111:0/4232173366 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558007cb0 con 0x7fb560145350 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 -- 172.21.15.111:0/4232173366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560145350 msgr2=0x7fb56006c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 --2- 172.21.15.111:0/4232173366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560145350 0x7fb56006c830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb56015fb30 tx=0x7fb55802a680 comp rx=0 tx=0).stop 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 -- 172.21.15.111:0/4232173366 shutdown_connections 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 --2- 172.21.15.111:0/4232173366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560145350 0x7fb56006c830 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 --2- 172.21.15.111:0/4232173366 >> v2:172.21.15.153:3300/0 conn(0x7fb5600b5e40 0x7fb56006f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 -- 172.21.15.111:0/4232173366 >> 172.21.15.111:0/4232173366 conn(0x7fb560073130 msgr2=0x7fb5600b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:34.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 -- 172.21.15.111:0/4232173366 shutdown_connections 2024-05-27T00:40:34.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.838+0000 7fb568370640 1 -- 172.21.15.111:0/4232173366 wait complete. 2024-05-27T00:40:34.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.839+0000 7fb568370640 1 Processor -- start 2024-05-27T00:40:34.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.839+0000 7fb568370640 1 -- start start 2024-05-27T00:40:34.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb568370640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5600b5e40 0x7fb56013d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb568370640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 0x7fb56013d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb568370640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb560145350 0x7fb5600ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb568370640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb560145760 con 0x7fb560144cd0 2024-05-27T00:40:34.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb568370640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601450b0 con 0x7fb5600b5e40 2024-05-27T00:40:34.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb568370640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb560160990 con 0x7fb560145350 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 0x7fb56013d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 0x7fb56013d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59496/0 (socket says 172.21.15.111:59496) 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 -- 172.21.15.111:0/3983323683 learned_addr learned my addr 172.21.15.111:0/3983323683 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56610c640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5600b5e40 0x7fb56013d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 -- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb560145350 msgr2=0x7fb5600ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56690d640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb560145350 0x7fb5600ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb560145350 0x7fb5600ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 -- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5600b5e40 msgr2=0x7fb56013d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5600b5e40 0x7fb56013d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 -- 172.21.15.111:0/3983323683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5601443d0 con 0x7fb560144cd0 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56610c640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5600b5e40 0x7fb56013d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb56590b640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 0x7fb56013d9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb55000c7f0 tx=0x7fb55000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.840+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550015070 con 0x7fb560144cd0 2024-05-27T00:40:34.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.841+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb550004240 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.841+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5600a9e10 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.841+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550007620 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.841+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5600aa3d0 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.841+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb560068680 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.842+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5500077c0 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.842+0000 7fb556ffd640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb54406be30 0x7fb54406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.842+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb55008b2a0 con 0x7fb560144cd0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.843+0000 7fb56610c640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb54406be30 0x7fb54406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:34.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.843+0000 7fb56610c640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb54406be30 0x7fb54406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb55c000f60 tx=0x7fb55c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:34.847 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:34.847+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb550059ac0 con 0x7fb560144cd0 2024-05-27T00:40:35.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.015+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5600b4370 con 0x7fb54406be30 2024-05-27T00:40:35.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.022+0000 7fb556ffd640 1 -- 172.21.15.111:0/3983323683 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fb5600b4370 con 0x7fb54406be30 2024-05-27T00:40:35.028 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:35.030 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:40:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb54406be30 msgr2=0x7fb54406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb54406be30 0x7fb54406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb55c000f60 tx=0x7fb55c009040 comp rx=0 tx=0).stop 2024-05-27T00:40:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 msgr2=0x7fb56013d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 0x7fb56013d9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb55000c7f0 tx=0x7fb55000ccb0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 shutdown_connections 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb54406be30 0x7fb54406e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb560145350 0x7fb5600ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb560144cd0 0x7fb56013d9c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 --2- 172.21.15.111:0/3983323683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5600b5e40 0x7fb56013d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 >> 172.21.15.111:0/3983323683 conn(0x7fb560073130 msgr2=0x7fb56006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 shutdown_connections 2024-05-27T00:40:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.025+0000 7fb568370640 1 -- 172.21.15.111:0/3983323683 wait complete. 2024-05-27T00:40:35.037 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-05-27T00:40:34.943007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220888,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753832,"statfs":{"total":773094113280,"available":772867923968,"internally_reserved":0,"allocated":4820992,"data_stored":2744096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001379"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846398+0000","last_change":"2024-05-27T00:40:30.855401+0000","last_active":"2024-05-27T00:40:31.846398+0000","last_peered":"2024-05-27T00:40:31.846398+0000","last_clean":"2024-05-27T00:40:31.846398+0000","last_became_active":"2024-05-27T00:40:30.852565+0000","last_became_peered":"2024-05-27T00:40:30.852565+0000","last_unstale":"2024-05-27T00:40:31.846398+0000","last_undegraded":"2024-05-27T00:40:31.846398+0000","last_fullsized":"2024-05-27T00:40:31.846398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:43.157586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846269+0000","last_change":"2024-05-27T00:40:30.855430+0000","last_active":"2024-05-27T00:40:31.846269+0000","last_peered":"2024-05-27T00:40:31.846269+0000","last_clean":"2024-05-27T00:40:31.846269+0000","last_became_active":"2024-05-27T00:40:30.855163+0000","last_became_peered":"2024-05-27T00:40:30.855163+0000","last_unstale":"2024-05-27T00:40:31.846269+0000","last_undegraded":"2024-05-27T00:40:31.846269+0000","last_fullsized":"2024-05-27T00:40:31.846269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:16.405801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755165+0000","last_change":"2024-05-27T00:40:30.854858+0000","last_active":"2024-05-27T00:40:33.755165+0000","last_peered":"2024-05-27T00:40:33.755165+0000","last_clean":"2024-05-27T00:40:33.755165+0000","last_became_active":"2024-05-27T00:40:30.854577+0000","last_became_peered":"2024-05-27T00:40:30.854577+0000","last_unstale":"2024-05-27T00:40:33.755165+0000","last_undegraded":"2024-05-27T00:40:33.755165+0000","last_fullsized":"2024-05-27T00:40:33.755165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755138+0000","last_change":"2024-05-27T00:40:30.852335+0000","last_active":"2024-05-27T00:40:33.755138+0000","last_peered":"2024-05-27T00:40:33.755138+0000","last_clean":"2024-05-27T00:40:33.755138+0000","last_became_active":"2024-05-27T00:40:30.852014+0000","last_became_peered":"2024-05-27T00:40:30.852014+0000","last_unstale":"2024-05-27T00:40:33.755138+0000","last_undegraded":"2024-05-27T00:40:33.755138+0000","last_fullsized":"2024-05-27T00:40:33.755138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:29:13.936412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754549+0000","last_change":"2024-05-27T00:40:30.847265+0000","last_active":"2024-05-27T00:40:33.754549+0000","last_peered":"2024-05-27T00:40:33.754549+0000","last_clean":"2024-05-27T00:40:33.754549+0000","last_became_active":"2024-05-27T00:40:30.846994+0000","last_became_peered":"2024-05-27T00:40:30.846994+0000","last_unstale":"2024-05-27T00:40:33.754549+0000","last_undegraded":"2024-05-27T00:40:33.754549+0000","last_fullsized":"2024-05-27T00:40:33.754549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:50:40.605562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844389+0000","last_change":"2024-05-27T00:40:30.854665+0000","last_active":"2024-05-27T00:40:31.844389+0000","last_peered":"2024-05-27T00:40:31.844389+0000","last_clean":"2024-05-27T00:40:31.844389+0000","last_became_active":"2024-05-27T00:40:30.854381+0000","last_became_peered":"2024-05-27T00:40:30.854381+0000","last_unstale":"2024-05-27T00:40:31.844389+0000","last_undegraded":"2024-05-27T00:40:31.844389+0000","last_fullsized":"2024-05-27T00:40:31.844389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:51.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963523+0000","last_change":"2024-05-27T00:40:30.853683+0000","last_active":"2024-05-27T00:40:31.963523+0000","last_peered":"2024-05-27T00:40:31.963523+0000","last_clean":"2024-05-27T00:40:31.963523+0000","last_became_active":"2024-05-27T00:40:30.853441+0000","last_became_peered":"2024-05-27T00:40:30.853441+0000","last_unstale":"2024-05-27T00:40:31.963523+0000","last_undegraded":"2024-05-27T00:40:31.963523+0000","last_fullsized":"2024-05-27T00:40:31.963523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846007+0000","last_change":"2024-05-27T00:40:30.855511+0000","last_active":"2024-05-27T00:40:31.846007+0000","last_peered":"2024-05-27T00:40:31.846007+0000","last_clean":"2024-05-27T00:40:31.846007+0000","last_became_active":"2024-05-27T00:40:30.854581+0000","last_became_peered":"2024-05-27T00:40:30.854581+0000","last_unstale":"2024-05-27T00:40:31.846007+0000","last_undegraded":"2024-05-27T00:40:31.846007+0000","last_fullsized":"2024-05-27T00:40:31.846007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:23:13.579222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755351+0000","last_change":"2024-05-27T00:40:30.854970+0000","last_active":"2024-05-27T00:40:33.755351+0000","last_peered":"2024-05-27T00:40:33.755351+0000","last_clean":"2024-05-27T00:40:33.755351+0000","last_became_active":"2024-05-27T00:40:30.854622+0000","last_became_peered":"2024-05-27T00:40:30.854622+0000","last_unstale":"2024-05-27T00:40:33.755351+0000","last_undegraded":"2024-05-27T00:40:33.755351+0000","last_fullsized":"2024-05-27T00:40:33.755351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963840+0000","last_change":"2024-05-27T00:40:30.842451+0000","last_active":"2024-05-27T00:40:31.963840+0000","last_peered":"2024-05-27T00:40:31.963840+0000","last_clean":"2024-05-27T00:40:31.963840+0000","last_became_active":"2024-05-27T00:40:30.841621+0000","last_became_peered":"2024-05-27T00:40:30.841621+0000","last_unstale":"2024-05-27T00:40:31.963840+0000","last_undegraded":"2024-05-27T00:40:31.963840+0000","last_fullsized":"2024-05-27T00:40:31.963840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:33:24.367596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964105+0000","last_change":"2024-05-27T00:40:30.847791+0000","last_active":"2024-05-27T00:40:31.964105+0000","last_peered":"2024-05-27T00:40:31.964105+0000","last_clean":"2024-05-27T00:40:31.964105+0000","last_became_active":"2024-05-27T00:40:30.847451+0000","last_became_peered":"2024-05-27T00:40:30.847451+0000","last_unstale":"2024-05-27T00:40:31.964105+0000","last_undegraded":"2024-05-27T00:40:31.964105+0000","last_fullsized":"2024-05-27T00:40:31.964105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:43:23.638293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755117+0000","last_change":"2024-05-27T00:40:30.842233+0000","last_active":"2024-05-27T00:40:33.755117+0000","last_peered":"2024-05-27T00:40:33.755117+0000","last_clean":"2024-05-27T00:40:33.755117+0000","last_became_active":"2024-05-27T00:40:30.841968+0000","last_became_peered":"2024-05-27T00:40:30.841968+0000","last_unstale":"2024-05-27T00:40:33.755117+0000","last_undegraded":"2024-05-27T00:40:33.755117+0000","last_fullsized":"2024-05-27T00:40:33.755117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963532+0000","last_change":"2024-05-27T00:40:30.842018+0000","last_active":"2024-05-27T00:40:31.963532+0000","last_peered":"2024-05-27T00:40:31.963532+0000","last_clean":"2024-05-27T00:40:31.963532+0000","last_became_active":"2024-05-27T00:40:30.841715+0000","last_became_peered":"2024-05-27T00:40:30.841715+0000","last_unstale":"2024-05-27T00:40:31.963532+0000","last_undegraded":"2024-05-27T00:40:31.963532+0000","last_fullsized":"2024-05-27T00:40:31.963532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:43:04.866508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755418+0000","last_change":"2024-05-27T00:40:30.856740+0000","last_active":"2024-05-27T00:40:33.755418+0000","last_peered":"2024-05-27T00:40:33.755418+0000","last_clean":"2024-05-27T00:40:33.755418+0000","last_became_active":"2024-05-27T00:40:30.856474+0000","last_became_peered":"2024-05-27T00:40:30.856474+0000","last_unstale":"2024-05-27T00:40:33.755418+0000","last_undegraded":"2024-05-27T00:40:33.755418+0000","last_fullsized":"2024-05-27T00:40:33.755418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963508+0000","last_change":"2024-05-27T00:40:30.845060+0000","last_active":"2024-05-27T00:40:31.963508+0000","last_peered":"2024-05-27T00:40:31.963508+0000","last_clean":"2024-05-27T00:40:31.963508+0000","last_became_active":"2024-05-27T00:40:30.844505+0000","last_became_peered":"2024-05-27T00:40:30.844505+0000","last_unstale":"2024-05-27T00:40:31.963508+0000","last_undegraded":"2024-05-27T00:40:31.963508+0000","last_fullsized":"2024-05-27T00:40:31.963508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755545+0000","last_change":"2024-05-27T00:40:30.851915+0000","last_active":"2024-05-27T00:40:33.755545+0000","last_peered":"2024-05-27T00:40:33.755545+0000","last_clean":"2024-05-27T00:40:33.755545+0000","last_became_active":"2024-05-27T00:40:30.851259+0000","last_became_peered":"2024-05-27T00:40:30.851259+0000","last_unstale":"2024-05-27T00:40:33.755545+0000","last_undegraded":"2024-05-27T00:40:33.755545+0000","last_fullsized":"2024-05-27T00:40:33.755545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755992+0000","last_change":"2024-05-27T00:40:30.851666+0000","last_active":"2024-05-27T00:40:33.755992+0000","last_peered":"2024-05-27T00:40:33.755992+0000","last_clean":"2024-05-27T00:40:33.755992+0000","last_became_active":"2024-05-27T00:40:30.851232+0000","last_became_peered":"2024-05-27T00:40:30.851232+0000","last_unstale":"2024-05-27T00:40:33.755992+0000","last_undegraded":"2024-05-27T00:40:33.755992+0000","last_fullsized":"2024-05-27T00:40:33.755992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:47:48.323260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847151+0000","last_change":"2024-05-27T00:40:30.848089+0000","last_active":"2024-05-27T00:40:31.847151+0000","last_peered":"2024-05-27T00:40:31.847151+0000","last_clean":"2024-05-27T00:40:31.847151+0000","last_became_active":"2024-05-27T00:40:30.847800+0000","last_became_peered":"2024-05-27T00:40:30.847800+0000","last_unstale":"2024-05-27T00:40:31.847151+0000","last_undegraded":"2024-05-27T00:40:31.847151+0000","last_fullsized":"2024-05-27T00:40:31.847151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:49:44.200574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755204+0000","last_change":"2024-05-27T00:40:30.845947+0000","last_active":"2024-05-27T00:40:33.755204+0000","last_peered":"2024-05-27T00:40:33.755204+0000","last_clean":"2024-05-27T00:40:33.755204+0000","last_became_active":"2024-05-27T00:40:30.845659+0000","last_became_peered":"2024-05-27T00:40:30.845659+0000","last_unstale":"2024-05-27T00:40:33.755204+0000","last_undegraded":"2024-05-27T00:40:33.755204+0000","last_fullsized":"2024-05-27T00:40:33.755204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:29.087885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846967+0000","last_change":"2024-05-27T00:40:30.853177+0000","last_active":"2024-05-27T00:40:31.846967+0000","last_peered":"2024-05-27T00:40:31.846967+0000","last_clean":"2024-05-27T00:40:31.846967+0000","last_became_active":"2024-05-27T00:40:30.852953+0000","last_became_peered":"2024-05-27T00:40:30.852953+0000","last_unstale":"2024-05-27T00:40:31.846967+0000","last_undegraded":"2024-05-27T00:40:31.846967+0000","last_fullsized":"2024-05-27T00:40:31.846967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:06.799224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963712+0000","last_change":"2024-05-27T00:40:30.844500+0000","last_active":"2024-05-27T00:40:31.963712+0000","last_peered":"2024-05-27T00:40:31.963712+0000","last_clean":"2024-05-27T00:40:31.963712+0000","last_became_active":"2024-05-27T00:40:30.844193+0000","last_became_peered":"2024-05-27T00:40:30.844193+0000","last_unstale":"2024-05-27T00:40:31.963712+0000","last_undegraded":"2024-05-27T00:40:31.963712+0000","last_fullsized":"2024-05-27T00:40:31.963712+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:51.877600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755742+0000","last_change":"2024-05-27T00:40:30.851942+0000","last_active":"2024-05-27T00:40:33.755742+0000","last_peered":"2024-05-27T00:40:33.755742+0000","last_clean":"2024-05-27T00:40:33.755742+0000","last_became_active":"2024-05-27T00:40:30.850621+0000","last_became_peered":"2024-05-27T00:40:30.850621+0000","last_unstale":"2024-05-27T00:40:33.755742+0000","last_undegraded":"2024-05-27T00:40:33.755742+0000","last_fullsized":"2024-05-27T00:40:33.755742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755649+0000","last_change":"2024-05-27T00:40:30.845599+0000","last_active":"2024-05-27T00:40:33.755649+0000","last_peered":"2024-05-27T00:40:33.755649+0000","last_clean":"2024-05-27T00:40:33.755649+0000","last_became_active":"2024-05-27T00:40:30.845119+0000","last_became_peered":"2024-05-27T00:40:30.845119+0000","last_unstale":"2024-05-27T00:40:33.755649+0000","last_undegraded":"2024-05-27T00:40:33.755649+0000","last_fullsized":"2024-05-27T00:40:33.755649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754550+0000","last_change":"2024-05-27T00:40:30.840661+0000","last_active":"2024-05-27T00:40:33.754550+0000","last_peered":"2024-05-27T00:40:33.754550+0000","last_clean":"2024-05-27T00:40:33.754550+0000","last_became_active":"2024-05-27T00:40:30.840388+0000","last_became_peered":"2024-05-27T00:40:30.840388+0000","last_unstale":"2024-05-27T00:40:33.754550+0000","last_undegraded":"2024-05-27T00:40:33.754550+0000","last_fullsized":"2024-05-27T00:40:33.754550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:06.004048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847108+0000","last_change":"2024-05-27T00:40:30.849378+0000","last_active":"2024-05-27T00:40:31.847108+0000","last_peered":"2024-05-27T00:40:31.847108+0000","last_clean":"2024-05-27T00:40:31.847108+0000","last_became_active":"2024-05-27T00:40:30.848917+0000","last_became_peered":"2024-05-27T00:40:30.848917+0000","last_unstale":"2024-05-27T00:40:31.847108+0000","last_undegraded":"2024-05-27T00:40:31.847108+0000","last_fullsized":"2024-05-27T00:40:31.847108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:56.990399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844659+0000","last_change":"2024-05-27T00:40:30.850361+0000","last_active":"2024-05-27T00:40:31.844659+0000","last_peered":"2024-05-27T00:40:31.844659+0000","last_clean":"2024-05-27T00:40:31.844659+0000","last_became_active":"2024-05-27T00:40:30.850080+0000","last_became_peered":"2024-05-27T00:40:30.850080+0000","last_unstale":"2024-05-27T00:40:31.844659+0000","last_undegraded":"2024-05-27T00:40:31.844659+0000","last_fullsized":"2024-05-27T00:40:31.844659+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847120+0000","last_change":"2024-05-27T00:40:30.842686+0000","last_active":"2024-05-27T00:40:31.847120+0000","last_peered":"2024-05-27T00:40:31.847120+0000","last_clean":"2024-05-27T00:40:31.847120+0000","last_became_active":"2024-05-27T00:40:30.842431+0000","last_became_peered":"2024-05-27T00:40:30.842431+0000","last_unstale":"2024-05-27T00:40:31.847120+0000","last_undegraded":"2024-05-27T00:40:31.847120+0000","last_fullsized":"2024-05-27T00:40:31.847120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:48.622372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963820+0000","last_change":"2024-05-27T00:40:30.856168+0000","last_active":"2024-05-27T00:40:31.963820+0000","last_peered":"2024-05-27T00:40:31.963820+0000","last_clean":"2024-05-27T00:40:31.963820+0000","last_became_active":"2024-05-27T00:40:30.855908+0000","last_became_peered":"2024-05-27T00:40:30.855908+0000","last_unstale":"2024-05-27T00:40:31.963820+0000","last_undegraded":"2024-05-27T00:40:31.963820+0000","last_fullsized":"2024-05-27T00:40:31.963820+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:37:28.064481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846932+0000","last_change":"2024-05-27T00:40:30.844268+0000","last_active":"2024-05-27T00:40:31.846932+0000","last_peered":"2024-05-27T00:40:31.846932+0000","last_clean":"2024-05-27T00:40:31.846932+0000","last_became_active":"2024-05-27T00:40:30.843982+0000","last_became_peered":"2024-05-27T00:40:30.843982+0000","last_unstale":"2024-05-27T00:40:31.846932+0000","last_undegraded":"2024-05-27T00:40:31.846932+0000","last_fullsized":"2024-05-27T00:40:31.846932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:03.839740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846947+0000","last_change":"2024-05-27T00:40:30.855497+0000","last_active":"2024-05-27T00:40:31.846947+0000","last_peered":"2024-05-27T00:40:31.846947+0000","last_clean":"2024-05-27T00:40:31.846947+0000","last_became_active":"2024-05-27T00:40:30.855231+0000","last_became_peered":"2024-05-27T00:40:30.855231+0000","last_unstale":"2024-05-27T00:40:31.846947+0000","last_undegraded":"2024-05-27T00:40:31.846947+0000","last_fullsized":"2024-05-27T00:40:31.846947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:26.292453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755889+0000","last_change":"2024-05-27T00:40:30.846422+0000","last_active":"2024-05-27T00:40:33.755889+0000","last_peered":"2024-05-27T00:40:33.755889+0000","last_clean":"2024-05-27T00:40:33.755889+0000","last_became_active":"2024-05-27T00:40:30.845272+0000","last_became_peered":"2024-05-27T00:40:30.845272+0000","last_unstale":"2024-05-27T00:40:33.755889+0000","last_undegraded":"2024-05-27T00:40:33.755889+0000","last_fullsized":"2024-05-27T00:40:33.755889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:08:49.450082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844307+0000","last_change":"2024-05-27T00:40:30.843789+0000","last_active":"2024-05-27T00:40:31.844307+0000","last_peered":"2024-05-27T00:40:31.844307+0000","last_clean":"2024-05-27T00:40:31.844307+0000","last_became_active":"2024-05-27T00:40:30.843473+0000","last_became_peered":"2024-05-27T00:40:30.843473+0000","last_unstale":"2024-05-27T00:40:31.844307+0000","last_undegraded":"2024-05-27T00:40:31.844307+0000","last_fullsized":"2024-05-27T00:40:31.844307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:37.783339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756208+0000","last_change":"2024-05-27T00:40:30.846361+0000","last_active":"2024-05-27T00:40:33.756208+0000","last_peered":"2024-05-27T00:40:33.756208+0000","last_clean":"2024-05-27T00:40:33.756208+0000","last_became_active":"2024-05-27T00:40:30.846099+0000","last_became_peered":"2024-05-27T00:40:30.846099+0000","last_unstale":"2024-05-27T00:40:33.756208+0000","last_undegraded":"2024-05-27T00:40:33.756208+0000","last_fullsized":"2024-05-27T00:40:33.756208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:43:04.866508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847082+0000","last_change":"2024-05-27T00:40:29.834920+0000","last_active":"2024-05-27T00:40:31.847082+0000","last_peered":"2024-05-27T00:40:31.847082+0000","last_clean":"2024-05-27T00:40:31.847082+0000","last_became_active":"2024-05-27T00:40:29.834624+0000","last_became_peered":"2024-05-27T00:40:29.834624+0000","last_unstale":"2024-05-27T00:40:31.847082+0000","last_undegraded":"2024-05-27T00:40:31.847082+0000","last_fullsized":"2024-05-27T00:40:31.847082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:50:39.603785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844620+0000","last_change":"2024-05-27T00:40:30.851854+0000","last_active":"2024-05-27T00:40:31.844620+0000","last_peered":"2024-05-27T00:40:31.844620+0000","last_clean":"2024-05-27T00:40:31.844620+0000","last_became_active":"2024-05-27T00:40:30.851553+0000","last_became_peered":"2024-05-27T00:40:30.851553+0000","last_unstale":"2024-05-27T00:40:31.844620+0000","last_undegraded":"2024-05-27T00:40:31.844620+0000","last_fullsized":"2024-05-27T00:40:31.844620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:12.844381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754975+0000","last_change":"2024-05-27T00:40:29.841137+0000","last_active":"2024-05-27T00:40:33.754975+0000","last_peered":"2024-05-27T00:40:33.754975+0000","last_clean":"2024-05-27T00:40:33.754975+0000","last_became_active":"2024-05-27T00:40:29.840779+0000","last_became_peered":"2024-05-27T00:40:29.840779+0000","last_unstale":"2024-05-27T00:40:33.754975+0000","last_undegraded":"2024-05-27T00:40:33.754975+0000","last_fullsized":"2024-05-27T00:40:33.754975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756351+0000","last_change":"2024-05-27T00:40:30.851783+0000","last_active":"2024-05-27T00:40:33.756351+0000","last_peered":"2024-05-27T00:40:33.756351+0000","last_clean":"2024-05-27T00:40:33.756351+0000","last_became_active":"2024-05-27T00:40:30.851513+0000","last_became_peered":"2024-05-27T00:40:30.851513+0000","last_unstale":"2024-05-27T00:40:33.756351+0000","last_undegraded":"2024-05-27T00:40:33.756351+0000","last_fullsized":"2024-05-27T00:40:33.756351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844676+0000","last_change":"2024-05-27T00:40:29.839689+0000","last_active":"2024-05-27T00:40:31.844676+0000","last_peered":"2024-05-27T00:40:31.844676+0000","last_clean":"2024-05-27T00:40:31.844676+0000","last_became_active":"2024-05-27T00:40:29.839397+0000","last_became_peered":"2024-05-27T00:40:29.839397+0000","last_unstale":"2024-05-27T00:40:31.844676+0000","last_undegraded":"2024-05-27T00:40:31.844676+0000","last_fullsized":"2024-05-27T00:40:31.844676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846352+0000","last_change":"2024-05-27T00:40:30.856094+0000","last_active":"2024-05-27T00:40:31.846352+0000","last_peered":"2024-05-27T00:40:31.846352+0000","last_clean":"2024-05-27T00:40:31.846352+0000","last_became_active":"2024-05-27T00:40:30.855830+0000","last_became_peered":"2024-05-27T00:40:30.855830+0000","last_unstale":"2024-05-27T00:40:31.846352+0000","last_undegraded":"2024-05-27T00:40:31.846352+0000","last_fullsized":"2024-05-27T00:40:31.846352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:20.414809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755950+0000","last_change":"2024-05-27T00:40:29.834650+0000","last_active":"2024-05-27T00:40:33.755950+0000","last_peered":"2024-05-27T00:40:33.755950+0000","last_clean":"2024-05-27T00:40:33.755950+0000","last_became_active":"2024-05-27T00:40:29.834368+0000","last_became_peered":"2024-05-27T00:40:29.834368+0000","last_unstale":"2024-05-27T00:40:33.755950+0000","last_undegraded":"2024-05-27T00:40:33.755950+0000","last_fullsized":"2024-05-27T00:40:33.755950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:55.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754551+0000","last_change":"2024-05-27T00:40:30.849028+0000","last_active":"2024-05-27T00:40:33.754551+0000","last_peered":"2024-05-27T00:40:33.754551+0000","last_clean":"2024-05-27T00:40:33.754551+0000","last_became_active":"2024-05-27T00:40:30.848705+0000","last_became_peered":"2024-05-27T00:40:30.848705+0000","last_unstale":"2024-05-27T00:40:33.754551+0000","last_undegraded":"2024-05-27T00:40:33.754551+0000","last_fullsized":"2024-05-27T00:40:33.754551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:37.783339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963613+0000","last_change":"2024-05-27T00:40:29.837136+0000","last_active":"2024-05-27T00:40:31.963613+0000","last_peered":"2024-05-27T00:40:31.963613+0000","last_clean":"2024-05-27T00:40:31.963613+0000","last_became_active":"2024-05-27T00:40:29.836334+0000","last_became_peered":"2024-05-27T00:40:29.836334+0000","last_unstale":"2024-05-27T00:40:31.963613+0000","last_undegraded":"2024-05-27T00:40:31.963613+0000","last_fullsized":"2024-05-27T00:40:31.963613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:55.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963773+0000","last_change":"2024-05-27T00:40:30.844814+0000","last_active":"2024-05-27T00:40:31.963773+0000","last_peered":"2024-05-27T00:40:31.963773+0000","last_clean":"2024-05-27T00:40:31.963773+0000","last_became_active":"2024-05-27T00:40:30.844140+0000","last_became_peered":"2024-05-27T00:40:30.844140+0000","last_unstale":"2024-05-27T00:40:31.963773+0000","last_undegraded":"2024-05-27T00:40:31.963773+0000","last_fullsized":"2024-05-27T00:40:31.963773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:09:09.478467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755478+0000","last_change":"2024-05-27T00:40:29.843655+0000","last_active":"2024-05-27T00:40:33.755478+0000","last_peered":"2024-05-27T00:40:33.755478+0000","last_clean":"2024-05-27T00:40:33.755478+0000","last_became_active":"2024-05-27T00:40:29.843287+0000","last_became_peered":"2024-05-27T00:40:29.843287+0000","last_unstale":"2024-05-27T00:40:33.755478+0000","last_undegraded":"2024-05-27T00:40:33.755478+0000","last_fullsized":"2024-05-27T00:40:33.755478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:28.086108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756101+0000","last_change":"2024-05-27T00:40:30.858629+0000","last_active":"2024-05-27T00:40:33.756101+0000","last_peered":"2024-05-27T00:40:33.756101+0000","last_clean":"2024-05-27T00:40:33.756101+0000","last_became_active":"2024-05-27T00:40:30.858341+0000","last_became_peered":"2024-05-27T00:40:30.858341+0000","last_unstale":"2024-05-27T00:40:33.756101+0000","last_undegraded":"2024-05-27T00:40:33.756101+0000","last_fullsized":"2024-05-27T00:40:33.756101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:08:49.450082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844334+0000","last_change":"2024-05-27T00:40:29.838981+0000","last_active":"2024-05-27T00:40:31.844334+0000","last_peered":"2024-05-27T00:40:31.844334+0000","last_clean":"2024-05-27T00:40:31.844334+0000","last_became_active":"2024-05-27T00:40:29.838673+0000","last_became_peered":"2024-05-27T00:40:29.838673+0000","last_unstale":"2024-05-27T00:40:31.844334+0000","last_undegraded":"2024-05-27T00:40:31.844334+0000","last_fullsized":"2024-05-27T00:40:31.844334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846176+0000","last_change":"2024-05-27T00:40:30.860388+0000","last_active":"2024-05-27T00:40:31.846176+0000","last_peered":"2024-05-27T00:40:31.846176+0000","last_clean":"2024-05-27T00:40:31.846176+0000","last_became_active":"2024-05-27T00:40:30.859958+0000","last_became_peered":"2024-05-27T00:40:30.859958+0000","last_unstale":"2024-05-27T00:40:31.846176+0000","last_undegraded":"2024-05-27T00:40:31.846176+0000","last_fullsized":"2024-05-27T00:40:31.846176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:37:28.064481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755183+0000","last_change":"2024-05-27T00:40:29.835914+0000","last_active":"2024-05-27T00:40:33.755183+0000","last_peered":"2024-05-27T00:40:33.755183+0000","last_clean":"2024-05-27T00:40:33.755183+0000","last_became_active":"2024-05-27T00:40:29.835645+0000","last_became_peered":"2024-05-27T00:40:29.835645+0000","last_unstale":"2024-05-27T00:40:33.755183+0000","last_undegraded":"2024-05-27T00:40:33.755183+0000","last_fullsized":"2024-05-27T00:40:33.755183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:25.290676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755251+0000","last_change":"2024-05-27T00:40:30.851583+0000","last_active":"2024-05-27T00:40:33.755251+0000","last_peered":"2024-05-27T00:40:33.755251+0000","last_clean":"2024-05-27T00:40:33.755251+0000","last_became_active":"2024-05-27T00:40:30.851320+0000","last_became_peered":"2024-05-27T00:40:30.851320+0000","last_unstale":"2024-05-27T00:40:33.755251+0000","last_undegraded":"2024-05-27T00:40:33.755251+0000","last_fullsized":"2024-05-27T00:40:33.755251+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:40.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846401+0000","last_change":"2024-05-27T00:40:29.839200+0000","last_active":"2024-05-27T00:40:31.846401+0000","last_peered":"2024-05-27T00:40:31.846401+0000","last_clean":"2024-05-27T00:40:31.846401+0000","last_became_active":"2024-05-27T00:40:29.838929+0000","last_became_peered":"2024-05-27T00:40:29.838929+0000","last_unstale":"2024-05-27T00:40:31.846401+0000","last_undegraded":"2024-05-27T00:40:31.846401+0000","last_fullsized":"2024-05-27T00:40:31.846401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:50.927711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755766+0000","last_change":"2024-05-27T00:40:30.843907+0000","last_active":"2024-05-27T00:40:33.755766+0000","last_peered":"2024-05-27T00:40:33.755766+0000","last_clean":"2024-05-27T00:40:33.755766+0000","last_became_active":"2024-05-27T00:40:30.842997+0000","last_became_peered":"2024-05-27T00:40:30.842997+0000","last_unstale":"2024-05-27T00:40:33.755766+0000","last_undegraded":"2024-05-27T00:40:33.755766+0000","last_fullsized":"2024-05-27T00:40:33.755766+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963897+0000","last_change":"2024-05-27T00:40:29.833098+0000","last_active":"2024-05-27T00:40:31.963897+0000","last_peered":"2024-05-27T00:40:31.963897+0000","last_clean":"2024-05-27T00:40:31.963897+0000","last_became_active":"2024-05-27T00:40:29.832815+0000","last_became_peered":"2024-05-27T00:40:29.832815+0000","last_unstale":"2024-05-27T00:40:31.963897+0000","last_undegraded":"2024-05-27T00:40:31.963897+0000","last_fullsized":"2024-05-27T00:40:31.963897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754923+0000","last_change":"2024-05-27T00:40:30.845131+0000","last_active":"2024-05-27T00:40:33.754923+0000","last_peered":"2024-05-27T00:40:33.754923+0000","last_clean":"2024-05-27T00:40:33.754923+0000","last_became_active":"2024-05-27T00:40:30.844824+0000","last_became_peered":"2024-05-27T00:40:30.844824+0000","last_unstale":"2024-05-27T00:40:33.754923+0000","last_undegraded":"2024-05-27T00:40:33.754923+0000","last_fullsized":"2024-05-27T00:40:33.754923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964014+0000","last_change":"2024-05-27T00:40:29.836855+0000","last_active":"2024-05-27T00:40:31.964014+0000","last_peered":"2024-05-27T00:40:31.964014+0000","last_clean":"2024-05-27T00:40:31.964014+0000","last_became_active":"2024-05-27T00:40:29.836413+0000","last_became_peered":"2024-05-27T00:40:29.836413+0000","last_unstale":"2024-05-27T00:40:31.964014+0000","last_undegraded":"2024-05-27T00:40:31.964014+0000","last_fullsized":"2024-05-27T00:40:31.964014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:47.620595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756225+0000","last_change":"2024-05-27T00:40:30.847851+0000","last_active":"2024-05-27T00:40:33.756225+0000","last_peered":"2024-05-27T00:40:33.756225+0000","last_clean":"2024-05-27T00:40:33.756225+0000","last_became_active":"2024-05-27T00:40:30.847595+0000","last_became_peered":"2024-05-27T00:40:30.847595+0000","last_unstale":"2024-05-27T00:40:33.756225+0000","last_undegraded":"2024-05-27T00:40:33.756225+0000","last_fullsized":"2024-05-27T00:40:33.756225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:47:48.323260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846276+0000","last_change":"2024-05-27T00:40:29.841007+0000","last_active":"2024-05-27T00:40:31.846276+0000","last_peered":"2024-05-27T00:40:31.846276+0000","last_clean":"2024-05-27T00:40:31.846276+0000","last_became_active":"2024-05-27T00:40:29.840694+0000","last_became_peered":"2024-05-27T00:40:29.840694+0000","last_unstale":"2024-05-27T00:40:31.846276+0000","last_undegraded":"2024-05-27T00:40:31.846276+0000","last_fullsized":"2024-05-27T00:40:31.846276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:55.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754714+0000","last_change":"2024-05-27T00:40:30.853848+0000","last_active":"2024-05-27T00:40:33.754714+0000","last_peered":"2024-05-27T00:40:33.754714+0000","last_clean":"2024-05-27T00:40:33.754714+0000","last_became_active":"2024-05-27T00:40:30.853608+0000","last_became_peered":"2024-05-27T00:40:30.853608+0000","last_unstale":"2024-05-27T00:40:33.754714+0000","last_undegraded":"2024-05-27T00:40:33.754714+0000","last_fullsized":"2024-05-27T00:40:33.754714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:56.389913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846275+0000","last_change":"2024-05-27T00:40:29.832617+0000","last_active":"2024-05-27T00:40:31.846275+0000","last_peered":"2024-05-27T00:40:31.846275+0000","last_clean":"2024-05-27T00:40:31.846275+0000","last_became_active":"2024-05-27T00:40:29.832366+0000","last_became_peered":"2024-05-27T00:40:29.832366+0000","last_unstale":"2024-05-27T00:40:31.846275+0000","last_undegraded":"2024-05-27T00:40:31.846275+0000","last_fullsized":"2024-05-27T00:40:31.846275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:11.842604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846092+0000","last_change":"2024-05-27T00:40:30.855023+0000","last_active":"2024-05-27T00:40:31.846092+0000","last_peered":"2024-05-27T00:40:31.846092+0000","last_clean":"2024-05-27T00:40:31.846092+0000","last_became_active":"2024-05-27T00:40:30.854711+0000","last_became_peered":"2024-05-27T00:40:30.854711+0000","last_unstale":"2024-05-27T00:40:31.846092+0000","last_undegraded":"2024-05-27T00:40:31.846092+0000","last_fullsized":"2024-05-27T00:40:31.846092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:09:09.478467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844453+0000","last_change":"2024-05-27T00:40:29.839332+0000","last_active":"2024-05-27T00:40:31.844453+0000","last_peered":"2024-05-27T00:40:31.844453+0000","last_clean":"2024-05-27T00:40:31.844453+0000","last_became_active":"2024-05-27T00:40:29.839058+0000","last_became_peered":"2024-05-27T00:40:29.839058+0000","last_unstale":"2024-05-27T00:40:31.844453+0000","last_undegraded":"2024-05-27T00:40:31.844453+0000","last_fullsized":"2024-05-27T00:40:31.844453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755413+0000","last_change":"2024-05-27T00:40:30.842022+0000","last_active":"2024-05-27T00:40:33.755413+0000","last_peered":"2024-05-27T00:40:33.755413+0000","last_clean":"2024-05-27T00:40:33.755413+0000","last_became_active":"2024-05-27T00:40:30.841565+0000","last_became_peered":"2024-05-27T00:40:30.841565+0000","last_unstale":"2024-05-27T00:40:33.755413+0000","last_undegraded":"2024-05-27T00:40:33.755413+0000","last_fullsized":"2024-05-27T00:40:33.755413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846503+0000","last_change":"2024-05-27T00:40:29.837660+0000","last_active":"2024-05-27T00:40:31.846503+0000","last_peered":"2024-05-27T00:40:31.846503+0000","last_clean":"2024-05-27T00:40:31.846503+0000","last_became_active":"2024-05-27T00:40:29.836617+0000","last_became_peered":"2024-05-27T00:40:29.836617+0000","last_unstale":"2024-05-27T00:40:31.846503+0000","last_undegraded":"2024-05-27T00:40:31.846503+0000","last_fullsized":"2024-05-27T00:40:31.846503+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:33.427371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846221+0000","last_change":"2024-05-27T00:40:30.852888+0000","last_active":"2024-05-27T00:40:31.846221+0000","last_peered":"2024-05-27T00:40:31.846221+0000","last_clean":"2024-05-27T00:40:31.846221+0000","last_became_active":"2024-05-27T00:40:30.852614+0000","last_became_peered":"2024-05-27T00:40:30.852614+0000","last_unstale":"2024-05-27T00:40:31.846221+0000","last_undegraded":"2024-05-27T00:40:31.846221+0000","last_fullsized":"2024-05-27T00:40:31.846221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:33:24.367596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755327+0000","last_change":"2024-05-27T00:40:29.844350+0000","last_active":"2024-05-27T00:40:33.755327+0000","last_peered":"2024-05-27T00:40:33.755327+0000","last_clean":"2024-05-27T00:40:33.755327+0000","last_became_active":"2024-05-27T00:40:29.843988+0000","last_became_peered":"2024-05-27T00:40:29.843988+0000","last_unstale":"2024-05-27T00:40:33.755327+0000","last_undegraded":"2024-05-27T00:40:33.755327+0000","last_fullsized":"2024-05-27T00:40:33.755327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846288+0000","last_change":"2024-05-27T00:40:30.860095+0000","last_active":"2024-05-27T00:40:31.846288+0000","last_peered":"2024-05-27T00:40:31.846288+0000","last_clean":"2024-05-27T00:40:31.846288+0000","last_became_active":"2024-05-27T00:40:30.858753+0000","last_became_peered":"2024-05-27T00:40:30.858753+0000","last_unstale":"2024-05-27T00:40:31.846288+0000","last_undegraded":"2024-05-27T00:40:31.846288+0000","last_fullsized":"2024-05-27T00:40:31.846288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:12.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963918+0000","last_change":"2024-05-27T00:40:29.836227+0000","last_active":"2024-05-27T00:40:31.963918+0000","last_peered":"2024-05-27T00:40:31.963918+0000","last_clean":"2024-05-27T00:40:31.963918+0000","last_became_active":"2024-05-27T00:40:29.835837+0000","last_became_peered":"2024-05-27T00:40:29.835837+0000","last_unstale":"2024-05-27T00:40:31.963918+0000","last_undegraded":"2024-05-27T00:40:31.963918+0000","last_fullsized":"2024-05-27T00:40:31.963918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:25.290676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963969+0000","last_change":"2024-05-27T00:40:30.848394+0000","last_active":"2024-05-27T00:40:31.963969+0000","last_peered":"2024-05-27T00:40:31.963969+0000","last_clean":"2024-05-27T00:40:31.963969+0000","last_became_active":"2024-05-27T00:40:30.846482+0000","last_became_peered":"2024-05-27T00:40:30.846482+0000","last_unstale":"2024-05-27T00:40:31.963969+0000","last_undegraded":"2024-05-27T00:40:31.963969+0000","last_fullsized":"2024-05-27T00:40:31.963969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:12.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756289+0000","last_change":"2024-05-27T00:40:29.838407+0000","last_active":"2024-05-27T00:40:33.756289+0000","last_peered":"2024-05-27T00:40:33.756289+0000","last_clean":"2024-05-27T00:40:33.756289+0000","last_became_active":"2024-05-27T00:40:29.837279+0000","last_became_peered":"2024-05-27T00:40:29.837279+0000","last_unstale":"2024-05-27T00:40:33.756289+0000","last_undegraded":"2024-05-27T00:40:33.756289+0000","last_fullsized":"2024-05-27T00:40:33.756289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:05.797447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756221+0000","last_change":"2024-05-27T00:40:30.846070+0000","last_active":"2024-05-27T00:40:33.756221+0000","last_peered":"2024-05-27T00:40:33.756221+0000","last_clean":"2024-05-27T00:40:33.756221+0000","last_became_active":"2024-05-27T00:40:30.845813+0000","last_became_peered":"2024-05-27T00:40:30.845813+0000","last_unstale":"2024-05-27T00:40:33.756221+0000","last_undegraded":"2024-05-27T00:40:33.756221+0000","last_fullsized":"2024-05-27T00:40:33.756221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755160+0000","last_change":"2024-05-27T00:40:29.836358+0000","last_active":"2024-05-27T00:40:33.755160+0000","last_peered":"2024-05-27T00:40:33.755160+0000","last_clean":"2024-05-27T00:40:33.755160+0000","last_became_active":"2024-05-27T00:40:29.836030+0000","last_became_peered":"2024-05-27T00:40:29.836030+0000","last_unstale":"2024-05-27T00:40:33.755160+0000","last_undegraded":"2024-05-27T00:40:33.755160+0000","last_fullsized":"2024-05-27T00:40:33.755160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:31.410996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754925+0000","last_change":"2024-05-27T00:40:30.858900+0000","last_active":"2024-05-27T00:40:33.754925+0000","last_peered":"2024-05-27T00:40:33.754925+0000","last_clean":"2024-05-27T00:40:33.754925+0000","last_became_active":"2024-05-27T00:40:30.858588+0000","last_became_peered":"2024-05-27T00:40:30.858588+0000","last_unstale":"2024-05-27T00:40:33.754925+0000","last_undegraded":"2024-05-27T00:40:33.754925+0000","last_fullsized":"2024-05-27T00:40:33.754925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:36.744482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844468+0000","last_change":"2024-05-27T00:40:29.843381+0000","last_active":"2024-05-27T00:40:31.844468+0000","last_peered":"2024-05-27T00:40:31.844468+0000","last_clean":"2024-05-27T00:40:31.844468+0000","last_became_active":"2024-05-27T00:40:29.843087+0000","last_became_peered":"2024-05-27T00:40:29.843087+0000","last_unstale":"2024-05-27T00:40:31.844468+0000","last_undegraded":"2024-05-27T00:40:31.844468+0000","last_fullsized":"2024-05-27T00:40:31.844468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:48.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755991+0000","last_change":"2024-05-27T00:40:30.844974+0000","last_active":"2024-05-27T00:40:33.755991+0000","last_peered":"2024-05-27T00:40:33.755991+0000","last_clean":"2024-05-27T00:40:33.755991+0000","last_became_active":"2024-05-27T00:40:30.843119+0000","last_became_peered":"2024-05-27T00:40:30.843119+0000","last_unstale":"2024-05-27T00:40:33.755991+0000","last_undegraded":"2024-05-27T00:40:33.755991+0000","last_fullsized":"2024-05-27T00:40:33.755991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754605+0000","last_change":"2024-05-27T00:40:29.831709+0000","last_active":"2024-05-27T00:40:33.754605+0000","last_peered":"2024-05-27T00:40:33.754605+0000","last_clean":"2024-05-27T00:40:33.754605+0000","last_became_active":"2024-05-27T00:40:29.831355+0000","last_became_peered":"2024-05-27T00:40:29.831355+0000","last_unstale":"2024-05-27T00:40:33.754605+0000","last_undegraded":"2024-05-27T00:40:33.754605+0000","last_fullsized":"2024-05-27T00:40:33.754605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:12.195311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754862+0000","last_change":"2024-05-27T00:40:30.844565+0000","last_active":"2024-05-27T00:40:33.754862+0000","last_peered":"2024-05-27T00:40:33.754862+0000","last_clean":"2024-05-27T00:40:33.754862+0000","last_became_active":"2024-05-27T00:40:30.844279+0000","last_became_peered":"2024-05-27T00:40:30.844279+0000","last_unstale":"2024-05-27T00:40:33.754862+0000","last_undegraded":"2024-05-27T00:40:33.754862+0000","last_fullsized":"2024-05-27T00:40:33.754862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:12.844381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754669+0000","last_change":"2024-05-27T00:40:29.835427+0000","last_active":"2024-05-27T00:40:33.754669+0000","last_peered":"2024-05-27T00:40:33.754669+0000","last_clean":"2024-05-27T00:40:33.754669+0000","last_became_active":"2024-05-27T00:40:29.834727+0000","last_became_peered":"2024-05-27T00:40:29.834727+0000","last_unstale":"2024-05-27T00:40:33.754669+0000","last_undegraded":"2024-05-27T00:40:33.754669+0000","last_fullsized":"2024-05-27T00:40:33.754669+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:58:42.956086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963716+0000","last_change":"2024-05-27T00:40:30.851594+0000","last_active":"2024-05-27T00:40:31.963716+0000","last_peered":"2024-05-27T00:40:31.963716+0000","last_clean":"2024-05-27T00:40:31.963716+0000","last_became_active":"2024-05-27T00:40:30.851318+0000","last_became_peered":"2024-05-27T00:40:30.851318+0000","last_unstale":"2024-05-27T00:40:31.963716+0000","last_undegraded":"2024-05-27T00:40:31.963716+0000","last_fullsized":"2024-05-27T00:40:31.963716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:23:13.579222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755911+0000","last_change":"2024-05-27T00:40:29.843742+0000","last_active":"2024-05-27T00:40:33.755911+0000","last_peered":"2024-05-27T00:40:33.755911+0000","last_clean":"2024-05-27T00:40:33.755911+0000","last_became_active":"2024-05-27T00:40:29.843445+0000","last_became_peered":"2024-05-27T00:40:29.843445+0000","last_unstale":"2024-05-27T00:40:33.755911+0000","last_undegraded":"2024-05-27T00:40:33.755911+0000","last_fullsized":"2024-05-27T00:40:33.755911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846350+0000","last_change":"2024-05-27T00:40:30.854867+0000","last_active":"2024-05-27T00:40:31.846350+0000","last_peered":"2024-05-27T00:40:31.846350+0000","last_clean":"2024-05-27T00:40:31.846350+0000","last_became_active":"2024-05-27T00:40:30.854579+0000","last_became_peered":"2024-05-27T00:40:30.854579+0000","last_unstale":"2024-05-27T00:40:31.846350+0000","last_undegraded":"2024-05-27T00:40:31.846350+0000","last_fullsized":"2024-05-27T00:40:31.846350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:06:27.438392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754435+0000","last_change":"2024-05-27T00:40:29.830333+0000","last_active":"2024-05-27T00:40:33.754435+0000","last_peered":"2024-05-27T00:40:33.754435+0000","last_clean":"2024-05-27T00:40:33.754435+0000","last_became_active":"2024-05-27T00:40:29.830050+0000","last_became_peered":"2024-05-27T00:40:29.830050+0000","last_unstale":"2024-05-27T00:40:33.754435+0000","last_undegraded":"2024-05-27T00:40:33.754435+0000","last_fullsized":"2024-05-27T00:40:33.754435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:01.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844646+0000","last_change":"2024-05-27T00:40:30.851081+0000","last_active":"2024-05-27T00:40:31.844646+0000","last_peered":"2024-05-27T00:40:31.844646+0000","last_clean":"2024-05-27T00:40:31.844646+0000","last_became_active":"2024-05-27T00:40:30.850755+0000","last_became_peered":"2024-05-27T00:40:30.850755+0000","last_unstale":"2024-05-27T00:40:31.844646+0000","last_undegraded":"2024-05-27T00:40:31.844646+0000","last_fullsized":"2024-05-27T00:40:31.844646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964070+0000","last_change":"2024-05-27T00:40:29.835772+0000","last_active":"2024-05-27T00:40:31.964070+0000","last_peered":"2024-05-27T00:40:31.964070+0000","last_clean":"2024-05-27T00:40:31.964070+0000","last_became_active":"2024-05-27T00:40:29.835453+0000","last_became_peered":"2024-05-27T00:40:29.835453+0000","last_unstale":"2024-05-27T00:40:31.964070+0000","last_undegraded":"2024-05-27T00:40:31.964070+0000","last_fullsized":"2024-05-27T00:40:31.964070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:05.797447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756274+0000","last_change":"2024-05-27T00:40:30.844109+0000","last_active":"2024-05-27T00:40:33.756274+0000","last_peered":"2024-05-27T00:40:33.756274+0000","last_clean":"2024-05-27T00:40:33.756274+0000","last_became_active":"2024-05-27T00:40:30.842384+0000","last_became_peered":"2024-05-27T00:40:30.842384+0000","last_unstale":"2024-05-27T00:40:33.756274+0000","last_undegraded":"2024-05-27T00:40:33.756274+0000","last_fullsized":"2024-05-27T00:40:33.756274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:51.877600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755046+0000","last_change":"2024-05-27T00:40:29.827858+0000","last_active":"2024-05-27T00:40:33.755046+0000","last_peered":"2024-05-27T00:40:33.755046+0000","last_clean":"2024-05-27T00:40:33.755046+0000","last_became_active":"2024-05-27T00:40:29.827589+0000","last_became_peered":"2024-05-27T00:40:29.827589+0000","last_unstale":"2024-05-27T00:40:33.755046+0000","last_undegraded":"2024-05-27T00:40:33.755046+0000","last_fullsized":"2024-05-27T00:40:33.755046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:28.730539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846318+0000","last_change":"2024-05-27T00:40:30.860768+0000","last_active":"2024-05-27T00:40:31.846318+0000","last_peered":"2024-05-27T00:40:31.846318+0000","last_clean":"2024-05-27T00:40:31.846318+0000","last_became_active":"2024-05-27T00:40:30.860451+0000","last_became_peered":"2024-05-27T00:40:30.860451+0000","last_unstale":"2024-05-27T00:40:31.846318+0000","last_undegraded":"2024-05-27T00:40:31.846318+0000","last_fullsized":"2024-05-27T00:40:31.846318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:43:23.638293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755169+0000","last_change":"2024-05-27T00:40:29.835183+0000","last_active":"2024-05-27T00:40:33.755169+0000","last_peered":"2024-05-27T00:40:33.755169+0000","last_clean":"2024-05-27T00:40:33.755169+0000","last_became_active":"2024-05-27T00:40:29.834922+0000","last_became_peered":"2024-05-27T00:40:29.834922+0000","last_unstale":"2024-05-27T00:40:33.755169+0000","last_undegraded":"2024-05-27T00:40:33.755169+0000","last_fullsized":"2024-05-27T00:40:33.755169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:28.086108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754715+0000","last_change":"2024-05-27T00:40:30.859747+0000","last_active":"2024-05-27T00:40:33.754715+0000","last_peered":"2024-05-27T00:40:33.754715+0000","last_clean":"2024-05-27T00:40:33.754715+0000","last_became_active":"2024-05-27T00:40:30.859488+0000","last_became_peered":"2024-05-27T00:40:30.859488+0000","last_unstale":"2024-05-27T00:40:33.754715+0000","last_undegraded":"2024-05-27T00:40:33.754715+0000","last_fullsized":"2024-05-27T00:40:33.754715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:51.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754941+0000","last_change":"2024-05-27T00:40:29.837690+0000","last_active":"2024-05-27T00:40:33.754941+0000","last_peered":"2024-05-27T00:40:33.754941+0000","last_clean":"2024-05-27T00:40:33.754941+0000","last_became_active":"2024-05-27T00:40:29.837411+0000","last_became_peered":"2024-05-27T00:40:29.837411+0000","last_unstale":"2024-05-27T00:40:33.754941+0000","last_undegraded":"2024-05-27T00:40:33.754941+0000","last_fullsized":"2024-05-27T00:40:33.754941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844355+0000","last_change":"2024-05-27T00:40:30.849204+0000","last_active":"2024-05-27T00:40:31.844355+0000","last_peered":"2024-05-27T00:40:31.844355+0000","last_clean":"2024-05-27T00:40:31.844355+0000","last_became_active":"2024-05-27T00:40:30.848866+0000","last_became_peered":"2024-05-27T00:40:30.848866+0000","last_unstale":"2024-05-27T00:40:31.844355+0000","last_undegraded":"2024-05-27T00:40:31.844355+0000","last_fullsized":"2024-05-27T00:40:31.844355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:56.389913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846650+0000","last_change":"2024-05-27T00:40:29.833805+0000","last_active":"2024-05-27T00:40:31.846650+0000","last_peered":"2024-05-27T00:40:31.846650+0000","last_clean":"2024-05-27T00:40:31.846650+0000","last_became_active":"2024-05-27T00:40:29.833515+0000","last_became_peered":"2024-05-27T00:40:29.833515+0000","last_unstale":"2024-05-27T00:40:31.846650+0000","last_undegraded":"2024-05-27T00:40:31.846650+0000","last_fullsized":"2024-05-27T00:40:31.846650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:05.002271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755459+0000","last_change":"2024-05-27T00:40:30.854612+0000","last_active":"2024-05-27T00:40:33.755459+0000","last_peered":"2024-05-27T00:40:33.755459+0000","last_clean":"2024-05-27T00:40:33.755459+0000","last_became_active":"2024-05-27T00:40:30.854334+0000","last_became_peered":"2024-05-27T00:40:30.854334+0000","last_unstale":"2024-05-27T00:40:33.755459+0000","last_undegraded":"2024-05-27T00:40:33.755459+0000","last_fullsized":"2024-05-27T00:40:33.755459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755280+0000","last_change":"2024-05-27T00:40:29.834692+0000","last_active":"2024-05-27T00:40:33.755280+0000","last_peered":"2024-05-27T00:40:33.755280+0000","last_clean":"2024-05-27T00:40:33.755280+0000","last_became_active":"2024-05-27T00:40:29.834412+0000","last_became_peered":"2024-05-27T00:40:29.834412+0000","last_unstale":"2024-05-27T00:40:33.755280+0000","last_undegraded":"2024-05-27T00:40:33.755280+0000","last_fullsized":"2024-05-27T00:40:33.755280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756089+0000","last_change":"2024-05-27T00:40:30.845502+0000","last_active":"2024-05-27T00:40:33.756089+0000","last_peered":"2024-05-27T00:40:33.756089+0000","last_clean":"2024-05-27T00:40:33.756089+0000","last_became_active":"2024-05-27T00:40:30.843028+0000","last_became_peered":"2024-05-27T00:40:30.843028+0000","last_unstale":"2024-05-27T00:40:33.756089+0000","last_undegraded":"2024-05-27T00:40:33.756089+0000","last_fullsized":"2024-05-27T00:40:33.756089+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846039+0000","last_change":"2024-05-27T00:40:29.835197+0000","last_active":"2024-05-27T00:40:31.846039+0000","last_peered":"2024-05-27T00:40:31.846039+0000","last_clean":"2024-05-27T00:40:31.846039+0000","last_became_active":"2024-05-27T00:40:29.834907+0000","last_became_peered":"2024-05-27T00:40:29.834907+0000","last_unstale":"2024-05-27T00:40:31.846039+0000","last_undegraded":"2024-05-27T00:40:31.846039+0000","last_fullsized":"2024-05-27T00:40:31.846039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:36.781562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963924+0000","last_change":"2024-05-27T00:40:30.859603+0000","last_active":"2024-05-27T00:40:31.963924+0000","last_peered":"2024-05-27T00:40:31.963924+0000","last_clean":"2024-05-27T00:40:31.963924+0000","last_became_active":"2024-05-27T00:40:30.859333+0000","last_became_peered":"2024-05-27T00:40:30.859333+0000","last_unstale":"2024-05-27T00:40:31.963924+0000","last_undegraded":"2024-05-27T00:40:31.963924+0000","last_fullsized":"2024-05-27T00:40:31.963924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:16.405801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756215+0000","last_change":"2024-05-27T00:40:29.843470+0000","last_active":"2024-05-27T00:40:33.756215+0000","last_peered":"2024-05-27T00:40:33.756215+0000","last_clean":"2024-05-27T00:40:33.756215+0000","last_became_active":"2024-05-27T00:40:29.843159+0000","last_became_peered":"2024-05-27T00:40:29.843159+0000","last_unstale":"2024-05-27T00:40:33.756215+0000","last_undegraded":"2024-05-27T00:40:33.756215+0000","last_fullsized":"2024-05-27T00:40:33.756215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:48.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755746+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:40:33.755746+0000","last_peered":"2024-05-27T00:40:33.755746+0000","last_clean":"2024-05-27T00:40:33.755746+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:40:33.755746+0000","last_undegraded":"2024-05-27T00:40:33.755746+0000","last_fullsized":"2024-05-27T00:40:33.755746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":197909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":197439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":198973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":197417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":781037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:40:35.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-27T00:40:35.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.175+0000 7f4ebfb61640 1 Processor -- start 2024-05-27T00:40:35.176 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.175+0000 7f4ebfb61640 1 -- start start 2024-05-27T00:40:35.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebfb61640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebfb61640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4eb80cce00 0x7f4eb8141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebfb61640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb8142650 con 0x7f4eb813ef60 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebfb61640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb80cf210 con 0x7f4eb80cce00 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebfb61640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80cebc0 con 0x7f4eb813f340 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebe0fe640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4eb80cce00 0x7f4eb8141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebd8fd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4eb813ef60 0x7f4eb80cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58986/0 (socket says 172.21.15.111:58986) 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebd8fd640 1 -- 172.21.15.111:0/3576865748 learned_addr learned my addr 172.21.15.111:0/3576865748 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebe0fe640 1 --2- 172.21.15.111:0/3576865748 >> v2:172.21.15.111:3300/0 conn(0x7f4eb80cce00 0x7f4eb8141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59506/0 (socket says 172.21.15.111:59506) 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebd0fc640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebe0fe640 1 -- 172.21.15.111:0/3576865748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 msgr2=0x7f4eb80cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebe0fe640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb80cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebe0fe640 1 -- 172.21.15.111:0/3576865748 >> v1:172.21.15.111:6789/0 conn(0x7f4eb813ef60 legacy=0x7f4eb80cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebc8fb640 1 -- 172.21.15.111:0/3576865748 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801598791 0 0) 0x7f4eb8142650 con 0x7f4eb813ef60 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebe0fe640 1 -- 172.21.15.111:0/3576865748 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80ceea0 con 0x7f4eb80cce00 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.176+0000 7f4ebd0fc640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.177+0000 7f4ebe0fe640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb80cce00 0x7f4eb8141920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009d90 tx=0x7f4eb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfee9e6bd624933 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.177+0000 7f4ebc8fb640 1 -- 172.21.15.111:0/3576865748 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007740 con 0x7f4eb80cce00 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.177+0000 7f4ebc8fb640 1 -- 172.21.15.111:0/3576865748 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb00078e0 con 0x7f4eb80cce00 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.177+0000 7f4ebc8fb640 1 -- 172.21.15.111:0/3576865748 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007cb0 con 0x7f4eb80cce00 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 -- 172.21.15.111:0/3576865748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb80cce00 msgr2=0x7f4eb8141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb80cce00 0x7f4eb8141920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009d90 tx=0x7f4eb002a660 comp rx=0 tx=0).stop 2024-05-27T00:40:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 -- 172.21.15.111:0/3576865748 shutdown_connections 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb80cce00 0x7f4eb8141920 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/3576865748 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb80cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 -- 172.21.15.111:0/3576865748 >> 172.21.15.111:0/3576865748 conn(0x7f4eb80b1c00 msgr2=0x7f4eb80b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 -- 172.21.15.111:0/3576865748 shutdown_connections 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.178+0000 7f4ebfb61640 1 -- 172.21.15.111:0/3576865748 wait complete. 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.179+0000 7f4ebfb61640 1 Processor -- start 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.179+0000 7f4ebfb61640 1 -- start start 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebfb61640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eb80cce00 0x7f4eb813e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebfb61640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 0x7f4eb815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebfb61640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb81f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebfb61640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8141fe0 con 0x7f4eb813ef60 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebfb61640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8141e60 con 0x7f4eb80cce00 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebfb61640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb8142160 con 0x7f4eb813f340 2024-05-27T00:40:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 0x7f4eb815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 0x7f4eb815ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59514/0 (socket says 172.21.15.111:59514) 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 -- 172.21.15.111:0/1977995539 learned_addr learned my addr 172.21.15.111:0/1977995539 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd8fd640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eb80cce00 0x7f4eb813e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebe0fe640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb81f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 -- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 msgr2=0x7f4eb81f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb81f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 -- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eb80cce00 msgr2=0x7f4eb813e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eb80cce00 0x7f4eb813e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 -- 172.21.15.111:0/1977995539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80b3370 con 0x7f4eb813ef60 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebe0fe640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb81f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd8fd640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eb80cce00 0x7f4eb813e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4ebd0fc640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 0x7f4eb815ade0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c7f0 tx=0x7f4ea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8015070 con 0x7f4eb813ef60 2024-05-27T00:40:35.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8007820 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.180+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8007bd0 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.181+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb80b3650 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.181+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb80b3c10 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.181+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb8068680 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.182+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ea8018070 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.182+0000 7f4eae7fc640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4e9c06be30 0x7f4e9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.183+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4ea808b0d0 con 0x7f4eb813ef60 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.183+0000 7f4ebd8fd640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4e9c06be30 0x7f4e9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.183+0000 7f4ebd8fd640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4e9c06be30 0x7f4e9c06e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4000f60 tx=0x7f4eb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:35.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.187+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4ea8059980 con 0x7f4eb813ef60 2024-05-27T00:40:35.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.426+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4eb806cc80 con 0x7f4eb813ef60 2024-05-27T00:40:35.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.427+0000 7f4eae7fc640 1 -- 172.21.15.111:0/1977995539 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1173 (secure 0 0 0) 0x7f4ea8059280 con 0x7f4eb813ef60 2024-05-27T00:40:35.428 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:35.428 INFO:teuthology.orchestra.run.smithi111.stdout:{"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":97,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1716770347,"num_in_osds":8,"osd_in_since":1716770338,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":226189312,"bytes_avail":772867923968,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5418}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-27T00:40:16.939246+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4e9c06be30 msgr2=0x7f4e9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4e9c06be30 0x7f4e9c06e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4000f60 tx=0x7f4eb4009040 comp rx=0 tx=0).stop 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 msgr2=0x7f4eb815ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 0x7f4eb815ade0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800c7f0 tx=0x7f4ea800ccb0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 shutdown_connections 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f4e9c06be30 0x7f4e9c06e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f4eb813f340 0x7f4eb81f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4eb813ef60 0x7f4eb815ade0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 --2- 172.21.15.111:0/1977995539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4eb80cce00 0x7f4eb813e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 >> 172.21.15.111:0/1977995539 conn(0x7f4eb80b1c00 msgr2=0x7f4eb80d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 shutdown_connections 2024-05-27T00:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.429+0000 7f4ebfb61640 1 -- 172.21.15.111:0/1977995539 wait complete. 2024-05-27T00:40:35.441 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-27T00:40:35.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:40:35.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.540+0000 7fdf4a807640 1 Processor -- start 2024-05-27T00:40:35.541 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.540+0000 7fdf4a807640 1 -- start start 2024-05-27T00:40:35.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf4a807640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413f340 0x7fdf440cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf4a807640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf440cce00 0x7fdf44141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf4a807640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf44142650 con 0x7fdf4413ef60 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf4a807640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fdf440cf210 con 0x7fdf440cce00 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf4a807640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdf440cebc0 con 0x7fdf4413f340 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf440cce00 0x7fdf44141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdf440cce00 0x7fdf44141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59520/0 (socket says 172.21.15.111:59520) 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 -- 172.21.15.111:0/2605700706 learned_addr learned my addr 172.21.15.111:0/2605700706 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf437fe640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413f340 0x7fdf440cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 -- 172.21.15.111:0/2605700706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413f340 msgr2=0x7fdf440cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413f340 0x7fdf440cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 -- 172.21.15.111:0/2605700706 >> v1:172.21.15.153:6789/0 conn(0x7fdf4413ef60 legacy=0x7fdf440cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf48da4640 1 -- 172.21.15.111:0/2605700706 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf440ceea0 con 0x7fdf440cce00 2024-05-27T00:40:35.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.541+0000 7fdf437fe640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413f340 0x7fdf440cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.542+0000 7fdf48da4640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf440cce00 0x7fdf44141920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e50 tx=0x7fdf3c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc2d21317d3d3fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.542+0000 7fdf42ffd640 1 -- 172.21.15.111:0/2605700706 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c036070 con 0x7fdf440cce00 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.542+0000 7fdf42ffd640 1 -- 172.21.15.111:0/2605700706 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c031600 con 0x7fdf440cce00 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.542+0000 7fdf42ffd640 1 -- 172.21.15.111:0/2605700706 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0319d0 con 0x7fdf440cce00 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 -- 172.21.15.111:0/2605700706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf440cce00 msgr2=0x7fdf44141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf440cce00 0x7fdf44141920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e50 tx=0x7fdf3c02a710 comp rx=0 tx=0).stop 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 -- 172.21.15.111:0/2605700706 shutdown_connections 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf440cce00 0x7fdf44141920 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 --2- 172.21.15.111:0/2605700706 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413f340 0x7fdf440cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 -- 172.21.15.111:0/2605700706 >> 172.21.15.111:0/2605700706 conn(0x7fdf440b1c00 msgr2=0x7fdf440b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:35.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 -- 172.21.15.111:0/2605700706 shutdown_connections 2024-05-27T00:40:35.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.543+0000 7fdf4a807640 1 -- 172.21.15.111:0/2605700706 wait complete. 2024-05-27T00:40:35.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.544+0000 7fdf4a807640 1 Processor -- start 2024-05-27T00:40:35.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.544+0000 7fdf4a807640 1 -- start start 2024-05-27T00:40:35.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf4a807640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf440cce00 0x7fdf44134610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf4a807640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413ef60 0x7fdf440721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf4a807640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 0x7fdf440726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf4a807640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44141fe0 con 0x7fdf4413f340 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf4a807640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44141e60 con 0x7fdf440cce00 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf4a807640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdf44142160 con 0x7fdf4413ef60 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 0x7fdf440726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 0x7fdf440726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59530/0 (socket says 172.21.15.111:59530) 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 -- 172.21.15.111:0/4142900319 learned_addr learned my addr 172.21.15.111:0/4142900319 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf43fff640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf440cce00 0x7fdf44134610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf437fe640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413ef60 0x7fdf440721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 -- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413ef60 msgr2=0x7fdf440721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413ef60 0x7fdf440721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 -- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf440cce00 msgr2=0x7fdf44134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf440cce00 0x7fdf44134610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.545+0000 7fdf48da4640 1 -- 172.21.15.111:0/4142900319 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf441429a0 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.546+0000 7fdf48da4640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 0x7fdf440726f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e50 tx=0x7fdf3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.546+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c036050 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.546+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c042070 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.546+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf44142c80 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.546+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c009510 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.546+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf44143240 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.547+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf44068680 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.547+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf3c007bd0 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.548+0000 7fdf40ff9640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdf3006be30 0x7fdf3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.548+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdf3c0abc50 con 0x7fdf4413f340 2024-05-27T00:40:35.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.548+0000 7fdf43fff640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdf3006be30 0x7fdf3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:35.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.548+0000 7fdf43fff640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdf3006be30 0x7fdf3006e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf340016e0 tx=0x7fdf34009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:35.553 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.552+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdf3c0b1020 con 0x7fdf4413f340 2024-05-27T00:40:35.575 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T00:40:35.575+0000 7faf33c19640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T00:40:35.723 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.723+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf4415c210 con 0x7fdf3006be30 2024-05-27T00:40:35.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.729+0000 7fdf40ff9640 1 -- 172.21.15.111:0/4142900319 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fdf4415c210 con 0x7fdf3006be30 2024-05-27T00:40:35.733 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:35.734 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:40:35.735 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdf3006be30 msgr2=0x7fdf3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdf3006be30 0x7fdf3006e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf340016e0 tx=0x7fdf34009040 comp rx=0 tx=0).stop 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 msgr2=0x7fdf440726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 0x7fdf440726f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e50 tx=0x7fdf3c004a40 comp rx=0 tx=0).stop 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 shutdown_connections 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdf3006be30 0x7fdf3006e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4413f340 0x7fdf440726f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdf4413ef60 0x7fdf440721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 --2- 172.21.15.111:0/4142900319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf440cce00 0x7fdf44134610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 >> 172.21.15.111:0/4142900319 conn(0x7fdf440b1c00 msgr2=0x7fdf4415e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 shutdown_connections 2024-05-27T00:40:35.736 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:35.732+0000 7fdf4a807640 1 -- 172.21.15.111:0/4142900319 wait complete. 2024-05-27T00:40:35.744 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-05-27T00:40:34.943007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220888,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753832,"statfs":{"total":773094113280,"available":772867923968,"internally_reserved":0,"allocated":4820992,"data_stored":2744096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001379"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846398+0000","last_change":"2024-05-27T00:40:30.855401+0000","last_active":"2024-05-27T00:40:31.846398+0000","last_peered":"2024-05-27T00:40:31.846398+0000","last_clean":"2024-05-27T00:40:31.846398+0000","last_became_active":"2024-05-27T00:40:30.852565+0000","last_became_peered":"2024-05-27T00:40:30.852565+0000","last_unstale":"2024-05-27T00:40:31.846398+0000","last_undegraded":"2024-05-27T00:40:31.846398+0000","last_fullsized":"2024-05-27T00:40:31.846398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:43.157586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846269+0000","last_change":"2024-05-27T00:40:30.855430+0000","last_active":"2024-05-27T00:40:31.846269+0000","last_peered":"2024-05-27T00:40:31.846269+0000","last_clean":"2024-05-27T00:40:31.846269+0000","last_became_active":"2024-05-27T00:40:30.855163+0000","last_became_peered":"2024-05-27T00:40:30.855163+0000","last_unstale":"2024-05-27T00:40:31.846269+0000","last_undegraded":"2024-05-27T00:40:31.846269+0000","last_fullsized":"2024-05-27T00:40:31.846269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:16.405801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755165+0000","last_change":"2024-05-27T00:40:30.854858+0000","last_active":"2024-05-27T00:40:33.755165+0000","last_peered":"2024-05-27T00:40:33.755165+0000","last_clean":"2024-05-27T00:40:33.755165+0000","last_became_active":"2024-05-27T00:40:30.854577+0000","last_became_peered":"2024-05-27T00:40:30.854577+0000","last_unstale":"2024-05-27T00:40:33.755165+0000","last_undegraded":"2024-05-27T00:40:33.755165+0000","last_fullsized":"2024-05-27T00:40:33.755165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755138+0000","last_change":"2024-05-27T00:40:30.852335+0000","last_active":"2024-05-27T00:40:33.755138+0000","last_peered":"2024-05-27T00:40:33.755138+0000","last_clean":"2024-05-27T00:40:33.755138+0000","last_became_active":"2024-05-27T00:40:30.852014+0000","last_became_peered":"2024-05-27T00:40:30.852014+0000","last_unstale":"2024-05-27T00:40:33.755138+0000","last_undegraded":"2024-05-27T00:40:33.755138+0000","last_fullsized":"2024-05-27T00:40:33.755138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:29:13.936412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754549+0000","last_change":"2024-05-27T00:40:30.847265+0000","last_active":"2024-05-27T00:40:33.754549+0000","last_peered":"2024-05-27T00:40:33.754549+0000","last_clean":"2024-05-27T00:40:33.754549+0000","last_became_active":"2024-05-27T00:40:30.846994+0000","last_became_peered":"2024-05-27T00:40:30.846994+0000","last_unstale":"2024-05-27T00:40:33.754549+0000","last_undegraded":"2024-05-27T00:40:33.754549+0000","last_fullsized":"2024-05-27T00:40:33.754549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:50:40.605562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844389+0000","last_change":"2024-05-27T00:40:30.854665+0000","last_active":"2024-05-27T00:40:31.844389+0000","last_peered":"2024-05-27T00:40:31.844389+0000","last_clean":"2024-05-27T00:40:31.844389+0000","last_became_active":"2024-05-27T00:40:30.854381+0000","last_became_peered":"2024-05-27T00:40:30.854381+0000","last_unstale":"2024-05-27T00:40:31.844389+0000","last_undegraded":"2024-05-27T00:40:31.844389+0000","last_fullsized":"2024-05-27T00:40:31.844389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:51.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963523+0000","last_change":"2024-05-27T00:40:30.853683+0000","last_active":"2024-05-27T00:40:31.963523+0000","last_peered":"2024-05-27T00:40:31.963523+0000","last_clean":"2024-05-27T00:40:31.963523+0000","last_became_active":"2024-05-27T00:40:30.853441+0000","last_became_peered":"2024-05-27T00:40:30.853441+0000","last_unstale":"2024-05-27T00:40:31.963523+0000","last_undegraded":"2024-05-27T00:40:31.963523+0000","last_fullsized":"2024-05-27T00:40:31.963523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846007+0000","last_change":"2024-05-27T00:40:30.855511+0000","last_active":"2024-05-27T00:40:31.846007+0000","last_peered":"2024-05-27T00:40:31.846007+0000","last_clean":"2024-05-27T00:40:31.846007+0000","last_became_active":"2024-05-27T00:40:30.854581+0000","last_became_peered":"2024-05-27T00:40:30.854581+0000","last_unstale":"2024-05-27T00:40:31.846007+0000","last_undegraded":"2024-05-27T00:40:31.846007+0000","last_fullsized":"2024-05-27T00:40:31.846007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:23:13.579222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755351+0000","last_change":"2024-05-27T00:40:30.854970+0000","last_active":"2024-05-27T00:40:33.755351+0000","last_peered":"2024-05-27T00:40:33.755351+0000","last_clean":"2024-05-27T00:40:33.755351+0000","last_became_active":"2024-05-27T00:40:30.854622+0000","last_became_peered":"2024-05-27T00:40:30.854622+0000","last_unstale":"2024-05-27T00:40:33.755351+0000","last_undegraded":"2024-05-27T00:40:33.755351+0000","last_fullsized":"2024-05-27T00:40:33.755351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963840+0000","last_change":"2024-05-27T00:40:30.842451+0000","last_active":"2024-05-27T00:40:31.963840+0000","last_peered":"2024-05-27T00:40:31.963840+0000","last_clean":"2024-05-27T00:40:31.963840+0000","last_became_active":"2024-05-27T00:40:30.841621+0000","last_became_peered":"2024-05-27T00:40:30.841621+0000","last_unstale":"2024-05-27T00:40:31.963840+0000","last_undegraded":"2024-05-27T00:40:31.963840+0000","last_fullsized":"2024-05-27T00:40:31.963840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:33:24.367596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964105+0000","last_change":"2024-05-27T00:40:30.847791+0000","last_active":"2024-05-27T00:40:31.964105+0000","last_peered":"2024-05-27T00:40:31.964105+0000","last_clean":"2024-05-27T00:40:31.964105+0000","last_became_active":"2024-05-27T00:40:30.847451+0000","last_became_peered":"2024-05-27T00:40:30.847451+0000","last_unstale":"2024-05-27T00:40:31.964105+0000","last_undegraded":"2024-05-27T00:40:31.964105+0000","last_fullsized":"2024-05-27T00:40:31.964105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:43:23.638293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755117+0000","last_change":"2024-05-27T00:40:30.842233+0000","last_active":"2024-05-27T00:40:33.755117+0000","last_peered":"2024-05-27T00:40:33.755117+0000","last_clean":"2024-05-27T00:40:33.755117+0000","last_became_active":"2024-05-27T00:40:30.841968+0000","last_became_peered":"2024-05-27T00:40:30.841968+0000","last_unstale":"2024-05-27T00:40:33.755117+0000","last_undegraded":"2024-05-27T00:40:33.755117+0000","last_fullsized":"2024-05-27T00:40:33.755117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963532+0000","last_change":"2024-05-27T00:40:30.842018+0000","last_active":"2024-05-27T00:40:31.963532+0000","last_peered":"2024-05-27T00:40:31.963532+0000","last_clean":"2024-05-27T00:40:31.963532+0000","last_became_active":"2024-05-27T00:40:30.841715+0000","last_became_peered":"2024-05-27T00:40:30.841715+0000","last_unstale":"2024-05-27T00:40:31.963532+0000","last_undegraded":"2024-05-27T00:40:31.963532+0000","last_fullsized":"2024-05-27T00:40:31.963532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:43:04.866508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755418+0000","last_change":"2024-05-27T00:40:30.856740+0000","last_active":"2024-05-27T00:40:33.755418+0000","last_peered":"2024-05-27T00:40:33.755418+0000","last_clean":"2024-05-27T00:40:33.755418+0000","last_became_active":"2024-05-27T00:40:30.856474+0000","last_became_peered":"2024-05-27T00:40:30.856474+0000","last_unstale":"2024-05-27T00:40:33.755418+0000","last_undegraded":"2024-05-27T00:40:33.755418+0000","last_fullsized":"2024-05-27T00:40:33.755418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963508+0000","last_change":"2024-05-27T00:40:30.845060+0000","last_active":"2024-05-27T00:40:31.963508+0000","last_peered":"2024-05-27T00:40:31.963508+0000","last_clean":"2024-05-27T00:40:31.963508+0000","last_became_active":"2024-05-27T00:40:30.844505+0000","last_became_peered":"2024-05-27T00:40:30.844505+0000","last_unstale":"2024-05-27T00:40:31.963508+0000","last_undegraded":"2024-05-27T00:40:31.963508+0000","last_fullsized":"2024-05-27T00:40:31.963508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755545+0000","last_change":"2024-05-27T00:40:30.851915+0000","last_active":"2024-05-27T00:40:33.755545+0000","last_peered":"2024-05-27T00:40:33.755545+0000","last_clean":"2024-05-27T00:40:33.755545+0000","last_became_active":"2024-05-27T00:40:30.851259+0000","last_became_peered":"2024-05-27T00:40:30.851259+0000","last_unstale":"2024-05-27T00:40:33.755545+0000","last_undegraded":"2024-05-27T00:40:33.755545+0000","last_fullsized":"2024-05-27T00:40:33.755545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755992+0000","last_change":"2024-05-27T00:40:30.851666+0000","last_active":"2024-05-27T00:40:33.755992+0000","last_peered":"2024-05-27T00:40:33.755992+0000","last_clean":"2024-05-27T00:40:33.755992+0000","last_became_active":"2024-05-27T00:40:30.851232+0000","last_became_peered":"2024-05-27T00:40:30.851232+0000","last_unstale":"2024-05-27T00:40:33.755992+0000","last_undegraded":"2024-05-27T00:40:33.755992+0000","last_fullsized":"2024-05-27T00:40:33.755992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:47:48.323260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847151+0000","last_change":"2024-05-27T00:40:30.848089+0000","last_active":"2024-05-27T00:40:31.847151+0000","last_peered":"2024-05-27T00:40:31.847151+0000","last_clean":"2024-05-27T00:40:31.847151+0000","last_became_active":"2024-05-27T00:40:30.847800+0000","last_became_peered":"2024-05-27T00:40:30.847800+0000","last_unstale":"2024-05-27T00:40:31.847151+0000","last_undegraded":"2024-05-27T00:40:31.847151+0000","last_fullsized":"2024-05-27T00:40:31.847151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:49:44.200574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755204+0000","last_change":"2024-05-27T00:40:30.845947+0000","last_active":"2024-05-27T00:40:33.755204+0000","last_peered":"2024-05-27T00:40:33.755204+0000","last_clean":"2024-05-27T00:40:33.755204+0000","last_became_active":"2024-05-27T00:40:30.845659+0000","last_became_peered":"2024-05-27T00:40:30.845659+0000","last_unstale":"2024-05-27T00:40:33.755204+0000","last_undegraded":"2024-05-27T00:40:33.755204+0000","last_fullsized":"2024-05-27T00:40:33.755204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:29.087885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846967+0000","last_change":"2024-05-27T00:40:30.853177+0000","last_active":"2024-05-27T00:40:31.846967+0000","last_peered":"2024-05-27T00:40:31.846967+0000","last_clean":"2024-05-27T00:40:31.846967+0000","last_became_active":"2024-05-27T00:40:30.852953+0000","last_became_peered":"2024-05-27T00:40:30.852953+0000","last_unstale":"2024-05-27T00:40:31.846967+0000","last_undegraded":"2024-05-27T00:40:31.846967+0000","last_fullsized":"2024-05-27T00:40:31.846967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:06.799224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963712+0000","last_change":"2024-05-27T00:40:30.844500+0000","last_active":"2024-05-27T00:40:31.963712+0000","last_peered":"2024-05-27T00:40:31.963712+0000","last_clean":"2024-05-27T00:40:31.963712+0000","last_became_active":"2024-05-27T00:40:30.844193+0000","last_became_peered":"2024-05-27T00:40:30.844193+0000","last_unstale":"2024-05-27T00:40:31.963712+0000","last_undegraded":"2024-05-27T00:40:31.963712+0000","last_fullsized":"2024-05-27T00:40:31.963712+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:51.877600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755742+0000","last_change":"2024-05-27T00:40:30.851942+0000","last_active":"2024-05-27T00:40:33.755742+0000","last_peered":"2024-05-27T00:40:33.755742+0000","last_clean":"2024-05-27T00:40:33.755742+0000","last_became_active":"2024-05-27T00:40:30.850621+0000","last_became_peered":"2024-05-27T00:40:30.850621+0000","last_unstale":"2024-05-27T00:40:33.755742+0000","last_undegraded":"2024-05-27T00:40:33.755742+0000","last_fullsized":"2024-05-27T00:40:33.755742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755649+0000","last_change":"2024-05-27T00:40:30.845599+0000","last_active":"2024-05-27T00:40:33.755649+0000","last_peered":"2024-05-27T00:40:33.755649+0000","last_clean":"2024-05-27T00:40:33.755649+0000","last_became_active":"2024-05-27T00:40:30.845119+0000","last_became_peered":"2024-05-27T00:40:30.845119+0000","last_unstale":"2024-05-27T00:40:33.755649+0000","last_undegraded":"2024-05-27T00:40:33.755649+0000","last_fullsized":"2024-05-27T00:40:33.755649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754550+0000","last_change":"2024-05-27T00:40:30.840661+0000","last_active":"2024-05-27T00:40:33.754550+0000","last_peered":"2024-05-27T00:40:33.754550+0000","last_clean":"2024-05-27T00:40:33.754550+0000","last_became_active":"2024-05-27T00:40:30.840388+0000","last_became_peered":"2024-05-27T00:40:30.840388+0000","last_unstale":"2024-05-27T00:40:33.754550+0000","last_undegraded":"2024-05-27T00:40:33.754550+0000","last_fullsized":"2024-05-27T00:40:33.754550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:06.004048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847108+0000","last_change":"2024-05-27T00:40:30.849378+0000","last_active":"2024-05-27T00:40:31.847108+0000","last_peered":"2024-05-27T00:40:31.847108+0000","last_clean":"2024-05-27T00:40:31.847108+0000","last_became_active":"2024-05-27T00:40:30.848917+0000","last_became_peered":"2024-05-27T00:40:30.848917+0000","last_unstale":"2024-05-27T00:40:31.847108+0000","last_undegraded":"2024-05-27T00:40:31.847108+0000","last_fullsized":"2024-05-27T00:40:31.847108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:56.990399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844659+0000","last_change":"2024-05-27T00:40:30.850361+0000","last_active":"2024-05-27T00:40:31.844659+0000","last_peered":"2024-05-27T00:40:31.844659+0000","last_clean":"2024-05-27T00:40:31.844659+0000","last_became_active":"2024-05-27T00:40:30.850080+0000","last_became_peered":"2024-05-27T00:40:30.850080+0000","last_unstale":"2024-05-27T00:40:31.844659+0000","last_undegraded":"2024-05-27T00:40:31.844659+0000","last_fullsized":"2024-05-27T00:40:31.844659+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847120+0000","last_change":"2024-05-27T00:40:30.842686+0000","last_active":"2024-05-27T00:40:31.847120+0000","last_peered":"2024-05-27T00:40:31.847120+0000","last_clean":"2024-05-27T00:40:31.847120+0000","last_became_active":"2024-05-27T00:40:30.842431+0000","last_became_peered":"2024-05-27T00:40:30.842431+0000","last_unstale":"2024-05-27T00:40:31.847120+0000","last_undegraded":"2024-05-27T00:40:31.847120+0000","last_fullsized":"2024-05-27T00:40:31.847120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:48.622372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963820+0000","last_change":"2024-05-27T00:40:30.856168+0000","last_active":"2024-05-27T00:40:31.963820+0000","last_peered":"2024-05-27T00:40:31.963820+0000","last_clean":"2024-05-27T00:40:31.963820+0000","last_became_active":"2024-05-27T00:40:30.855908+0000","last_became_peered":"2024-05-27T00:40:30.855908+0000","last_unstale":"2024-05-27T00:40:31.963820+0000","last_undegraded":"2024-05-27T00:40:31.963820+0000","last_fullsized":"2024-05-27T00:40:31.963820+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:37:28.064481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846932+0000","last_change":"2024-05-27T00:40:30.844268+0000","last_active":"2024-05-27T00:40:31.846932+0000","last_peered":"2024-05-27T00:40:31.846932+0000","last_clean":"2024-05-27T00:40:31.846932+0000","last_became_active":"2024-05-27T00:40:30.843982+0000","last_became_peered":"2024-05-27T00:40:30.843982+0000","last_unstale":"2024-05-27T00:40:31.846932+0000","last_undegraded":"2024-05-27T00:40:31.846932+0000","last_fullsized":"2024-05-27T00:40:31.846932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:03.839740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846947+0000","last_change":"2024-05-27T00:40:30.855497+0000","last_active":"2024-05-27T00:40:31.846947+0000","last_peered":"2024-05-27T00:40:31.846947+0000","last_clean":"2024-05-27T00:40:31.846947+0000","last_became_active":"2024-05-27T00:40:30.855231+0000","last_became_peered":"2024-05-27T00:40:30.855231+0000","last_unstale":"2024-05-27T00:40:31.846947+0000","last_undegraded":"2024-05-27T00:40:31.846947+0000","last_fullsized":"2024-05-27T00:40:31.846947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:26.292453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755889+0000","last_change":"2024-05-27T00:40:30.846422+0000","last_active":"2024-05-27T00:40:33.755889+0000","last_peered":"2024-05-27T00:40:33.755889+0000","last_clean":"2024-05-27T00:40:33.755889+0000","last_became_active":"2024-05-27T00:40:30.845272+0000","last_became_peered":"2024-05-27T00:40:30.845272+0000","last_unstale":"2024-05-27T00:40:33.755889+0000","last_undegraded":"2024-05-27T00:40:33.755889+0000","last_fullsized":"2024-05-27T00:40:33.755889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:08:49.450082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844307+0000","last_change":"2024-05-27T00:40:30.843789+0000","last_active":"2024-05-27T00:40:31.844307+0000","last_peered":"2024-05-27T00:40:31.844307+0000","last_clean":"2024-05-27T00:40:31.844307+0000","last_became_active":"2024-05-27T00:40:30.843473+0000","last_became_peered":"2024-05-27T00:40:30.843473+0000","last_unstale":"2024-05-27T00:40:31.844307+0000","last_undegraded":"2024-05-27T00:40:31.844307+0000","last_fullsized":"2024-05-27T00:40:31.844307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:37.783339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756208+0000","last_change":"2024-05-27T00:40:30.846361+0000","last_active":"2024-05-27T00:40:33.756208+0000","last_peered":"2024-05-27T00:40:33.756208+0000","last_clean":"2024-05-27T00:40:33.756208+0000","last_became_active":"2024-05-27T00:40:30.846099+0000","last_became_peered":"2024-05-27T00:40:30.846099+0000","last_unstale":"2024-05-27T00:40:33.756208+0000","last_undegraded":"2024-05-27T00:40:33.756208+0000","last_fullsized":"2024-05-27T00:40:33.756208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:43:04.866508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847082+0000","last_change":"2024-05-27T00:40:29.834920+0000","last_active":"2024-05-27T00:40:31.847082+0000","last_peered":"2024-05-27T00:40:31.847082+0000","last_clean":"2024-05-27T00:40:31.847082+0000","last_became_active":"2024-05-27T00:40:29.834624+0000","last_became_peered":"2024-05-27T00:40:29.834624+0000","last_unstale":"2024-05-27T00:40:31.847082+0000","last_undegraded":"2024-05-27T00:40:31.847082+0000","last_fullsized":"2024-05-27T00:40:31.847082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:50:39.603785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844620+0000","last_change":"2024-05-27T00:40:30.851854+0000","last_active":"2024-05-27T00:40:31.844620+0000","last_peered":"2024-05-27T00:40:31.844620+0000","last_clean":"2024-05-27T00:40:31.844620+0000","last_became_active":"2024-05-27T00:40:30.851553+0000","last_became_peered":"2024-05-27T00:40:30.851553+0000","last_unstale":"2024-05-27T00:40:31.844620+0000","last_undegraded":"2024-05-27T00:40:31.844620+0000","last_fullsized":"2024-05-27T00:40:31.844620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:12.844381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754975+0000","last_change":"2024-05-27T00:40:29.841137+0000","last_active":"2024-05-27T00:40:33.754975+0000","last_peered":"2024-05-27T00:40:33.754975+0000","last_clean":"2024-05-27T00:40:33.754975+0000","last_became_active":"2024-05-27T00:40:29.840779+0000","last_became_peered":"2024-05-27T00:40:29.840779+0000","last_unstale":"2024-05-27T00:40:33.754975+0000","last_undegraded":"2024-05-27T00:40:33.754975+0000","last_fullsized":"2024-05-27T00:40:33.754975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756351+0000","last_change":"2024-05-27T00:40:30.851783+0000","last_active":"2024-05-27T00:40:33.756351+0000","last_peered":"2024-05-27T00:40:33.756351+0000","last_clean":"2024-05-27T00:40:33.756351+0000","last_became_active":"2024-05-27T00:40:30.851513+0000","last_became_peered":"2024-05-27T00:40:30.851513+0000","last_unstale":"2024-05-27T00:40:33.756351+0000","last_undegraded":"2024-05-27T00:40:33.756351+0000","last_fullsized":"2024-05-27T00:40:33.756351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844676+0000","last_change":"2024-05-27T00:40:29.839689+0000","last_active":"2024-05-27T00:40:31.844676+0000","last_peered":"2024-05-27T00:40:31.844676+0000","last_clean":"2024-05-27T00:40:31.844676+0000","last_became_active":"2024-05-27T00:40:29.839397+0000","last_became_peered":"2024-05-27T00:40:29.839397+0000","last_unstale":"2024-05-27T00:40:31.844676+0000","last_undegraded":"2024-05-27T00:40:31.844676+0000","last_fullsized":"2024-05-27T00:40:31.844676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846352+0000","last_change":"2024-05-27T00:40:30.856094+0000","last_active":"2024-05-27T00:40:31.846352+0000","last_peered":"2024-05-27T00:40:31.846352+0000","last_clean":"2024-05-27T00:40:31.846352+0000","last_became_active":"2024-05-27T00:40:30.855830+0000","last_became_peered":"2024-05-27T00:40:30.855830+0000","last_unstale":"2024-05-27T00:40:31.846352+0000","last_undegraded":"2024-05-27T00:40:31.846352+0000","last_fullsized":"2024-05-27T00:40:31.846352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:20.414809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755950+0000","last_change":"2024-05-27T00:40:29.834650+0000","last_active":"2024-05-27T00:40:33.755950+0000","last_peered":"2024-05-27T00:40:33.755950+0000","last_clean":"2024-05-27T00:40:33.755950+0000","last_became_active":"2024-05-27T00:40:29.834368+0000","last_became_peered":"2024-05-27T00:40:29.834368+0000","last_unstale":"2024-05-27T00:40:33.755950+0000","last_undegraded":"2024-05-27T00:40:33.755950+0000","last_fullsized":"2024-05-27T00:40:33.755950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:55.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754551+0000","last_change":"2024-05-27T00:40:30.849028+0000","last_active":"2024-05-27T00:40:33.754551+0000","last_peered":"2024-05-27T00:40:33.754551+0000","last_clean":"2024-05-27T00:40:33.754551+0000","last_became_active":"2024-05-27T00:40:30.848705+0000","last_became_peered":"2024-05-27T00:40:30.848705+0000","last_unstale":"2024-05-27T00:40:33.754551+0000","last_undegraded":"2024-05-27T00:40:33.754551+0000","last_fullsized":"2024-05-27T00:40:33.754551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:37.783339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963613+0000","last_change":"2024-05-27T00:40:29.837136+0000","last_active":"2024-05-27T00:40:31.963613+0000","last_peered":"2024-05-27T00:40:31.963613+0000","last_clean":"2024-05-27T00:40:31.963613+0000","last_became_active":"2024-05-27T00:40:29.836334+0000","last_became_peered":"2024-05-27T00:40:29.836334+0000","last_unstale":"2024-05-27T00:40:31.963613+0000","last_undegraded":"2024-05-27T00:40:31.963613+0000","last_fullsized":"2024-05-27T00:40:31.963613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:55.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963773+0000","last_change":"2024-05-27T00:40:30.844814+0000","last_active":"2024-05-27T00:40:31.963773+0000","last_peered":"2024-05-27T00:40:31.963773+0000","last_clean":"2024-05-27T00:40:31.963773+0000","last_became_active":"2024-05-27T00:40:30.844140+0000","last_became_peered":"2024-05-27T00:40:30.844140+0000","last_unstale":"2024-05-27T00:40:31.963773+0000","last_undegraded":"2024-05-27T00:40:31.963773+0000","last_fullsized":"2024-05-27T00:40:31.963773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:09:09.478467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755478+0000","last_change":"2024-05-27T00:40:29.843655+0000","last_active":"2024-05-27T00:40:33.755478+0000","last_peered":"2024-05-27T00:40:33.755478+0000","last_clean":"2024-05-27T00:40:33.755478+0000","last_became_active":"2024-05-27T00:40:29.843287+0000","last_became_peered":"2024-05-27T00:40:29.843287+0000","last_unstale":"2024-05-27T00:40:33.755478+0000","last_undegraded":"2024-05-27T00:40:33.755478+0000","last_fullsized":"2024-05-27T00:40:33.755478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:28.086108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756101+0000","last_change":"2024-05-27T00:40:30.858629+0000","last_active":"2024-05-27T00:40:33.756101+0000","last_peered":"2024-05-27T00:40:33.756101+0000","last_clean":"2024-05-27T00:40:33.756101+0000","last_became_active":"2024-05-27T00:40:30.858341+0000","last_became_peered":"2024-05-27T00:40:30.858341+0000","last_unstale":"2024-05-27T00:40:33.756101+0000","last_undegraded":"2024-05-27T00:40:33.756101+0000","last_fullsized":"2024-05-27T00:40:33.756101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:08:49.450082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844334+0000","last_change":"2024-05-27T00:40:29.838981+0000","last_active":"2024-05-27T00:40:31.844334+0000","last_peered":"2024-05-27T00:40:31.844334+0000","last_clean":"2024-05-27T00:40:31.844334+0000","last_became_active":"2024-05-27T00:40:29.838673+0000","last_became_peered":"2024-05-27T00:40:29.838673+0000","last_unstale":"2024-05-27T00:40:31.844334+0000","last_undegraded":"2024-05-27T00:40:31.844334+0000","last_fullsized":"2024-05-27T00:40:31.844334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846176+0000","last_change":"2024-05-27T00:40:30.860388+0000","last_active":"2024-05-27T00:40:31.846176+0000","last_peered":"2024-05-27T00:40:31.846176+0000","last_clean":"2024-05-27T00:40:31.846176+0000","last_became_active":"2024-05-27T00:40:30.859958+0000","last_became_peered":"2024-05-27T00:40:30.859958+0000","last_unstale":"2024-05-27T00:40:31.846176+0000","last_undegraded":"2024-05-27T00:40:31.846176+0000","last_fullsized":"2024-05-27T00:40:31.846176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:37:28.064481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755183+0000","last_change":"2024-05-27T00:40:29.835914+0000","last_active":"2024-05-27T00:40:33.755183+0000","last_peered":"2024-05-27T00:40:33.755183+0000","last_clean":"2024-05-27T00:40:33.755183+0000","last_became_active":"2024-05-27T00:40:29.835645+0000","last_became_peered":"2024-05-27T00:40:29.835645+0000","last_unstale":"2024-05-27T00:40:33.755183+0000","last_undegraded":"2024-05-27T00:40:33.755183+0000","last_fullsized":"2024-05-27T00:40:33.755183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:25.290676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755251+0000","last_change":"2024-05-27T00:40:30.851583+0000","last_active":"2024-05-27T00:40:33.755251+0000","last_peered":"2024-05-27T00:40:33.755251+0000","last_clean":"2024-05-27T00:40:33.755251+0000","last_became_active":"2024-05-27T00:40:30.851320+0000","last_became_peered":"2024-05-27T00:40:30.851320+0000","last_unstale":"2024-05-27T00:40:33.755251+0000","last_undegraded":"2024-05-27T00:40:33.755251+0000","last_fullsized":"2024-05-27T00:40:33.755251+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:40.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846401+0000","last_change":"2024-05-27T00:40:29.839200+0000","last_active":"2024-05-27T00:40:31.846401+0000","last_peered":"2024-05-27T00:40:31.846401+0000","last_clean":"2024-05-27T00:40:31.846401+0000","last_became_active":"2024-05-27T00:40:29.838929+0000","last_became_peered":"2024-05-27T00:40:29.838929+0000","last_unstale":"2024-05-27T00:40:31.846401+0000","last_undegraded":"2024-05-27T00:40:31.846401+0000","last_fullsized":"2024-05-27T00:40:31.846401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:50.927711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755766+0000","last_change":"2024-05-27T00:40:30.843907+0000","last_active":"2024-05-27T00:40:33.755766+0000","last_peered":"2024-05-27T00:40:33.755766+0000","last_clean":"2024-05-27T00:40:33.755766+0000","last_became_active":"2024-05-27T00:40:30.842997+0000","last_became_peered":"2024-05-27T00:40:30.842997+0000","last_unstale":"2024-05-27T00:40:33.755766+0000","last_undegraded":"2024-05-27T00:40:33.755766+0000","last_fullsized":"2024-05-27T00:40:33.755766+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963897+0000","last_change":"2024-05-27T00:40:29.833098+0000","last_active":"2024-05-27T00:40:31.963897+0000","last_peered":"2024-05-27T00:40:31.963897+0000","last_clean":"2024-05-27T00:40:31.963897+0000","last_became_active":"2024-05-27T00:40:29.832815+0000","last_became_peered":"2024-05-27T00:40:29.832815+0000","last_unstale":"2024-05-27T00:40:31.963897+0000","last_undegraded":"2024-05-27T00:40:31.963897+0000","last_fullsized":"2024-05-27T00:40:31.963897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754923+0000","last_change":"2024-05-27T00:40:30.845131+0000","last_active":"2024-05-27T00:40:33.754923+0000","last_peered":"2024-05-27T00:40:33.754923+0000","last_clean":"2024-05-27T00:40:33.754923+0000","last_became_active":"2024-05-27T00:40:30.844824+0000","last_became_peered":"2024-05-27T00:40:30.844824+0000","last_unstale":"2024-05-27T00:40:33.754923+0000","last_undegraded":"2024-05-27T00:40:33.754923+0000","last_fullsized":"2024-05-27T00:40:33.754923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964014+0000","last_change":"2024-05-27T00:40:29.836855+0000","last_active":"2024-05-27T00:40:31.964014+0000","last_peered":"2024-05-27T00:40:31.964014+0000","last_clean":"2024-05-27T00:40:31.964014+0000","last_became_active":"2024-05-27T00:40:29.836413+0000","last_became_peered":"2024-05-27T00:40:29.836413+0000","last_unstale":"2024-05-27T00:40:31.964014+0000","last_undegraded":"2024-05-27T00:40:31.964014+0000","last_fullsized":"2024-05-27T00:40:31.964014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:47.620595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756225+0000","last_change":"2024-05-27T00:40:30.847851+0000","last_active":"2024-05-27T00:40:33.756225+0000","last_peered":"2024-05-27T00:40:33.756225+0000","last_clean":"2024-05-27T00:40:33.756225+0000","last_became_active":"2024-05-27T00:40:30.847595+0000","last_became_peered":"2024-05-27T00:40:30.847595+0000","last_unstale":"2024-05-27T00:40:33.756225+0000","last_undegraded":"2024-05-27T00:40:33.756225+0000","last_fullsized":"2024-05-27T00:40:33.756225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:47:48.323260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846276+0000","last_change":"2024-05-27T00:40:29.841007+0000","last_active":"2024-05-27T00:40:31.846276+0000","last_peered":"2024-05-27T00:40:31.846276+0000","last_clean":"2024-05-27T00:40:31.846276+0000","last_became_active":"2024-05-27T00:40:29.840694+0000","last_became_peered":"2024-05-27T00:40:29.840694+0000","last_unstale":"2024-05-27T00:40:31.846276+0000","last_undegraded":"2024-05-27T00:40:31.846276+0000","last_fullsized":"2024-05-27T00:40:31.846276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:55.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754714+0000","last_change":"2024-05-27T00:40:30.853848+0000","last_active":"2024-05-27T00:40:33.754714+0000","last_peered":"2024-05-27T00:40:33.754714+0000","last_clean":"2024-05-27T00:40:33.754714+0000","last_became_active":"2024-05-27T00:40:30.853608+0000","last_became_peered":"2024-05-27T00:40:30.853608+0000","last_unstale":"2024-05-27T00:40:33.754714+0000","last_undegraded":"2024-05-27T00:40:33.754714+0000","last_fullsized":"2024-05-27T00:40:33.754714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:56.389913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846275+0000","last_change":"2024-05-27T00:40:29.832617+0000","last_active":"2024-05-27T00:40:31.846275+0000","last_peered":"2024-05-27T00:40:31.846275+0000","last_clean":"2024-05-27T00:40:31.846275+0000","last_became_active":"2024-05-27T00:40:29.832366+0000","last_became_peered":"2024-05-27T00:40:29.832366+0000","last_unstale":"2024-05-27T00:40:31.846275+0000","last_undegraded":"2024-05-27T00:40:31.846275+0000","last_fullsized":"2024-05-27T00:40:31.846275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:11.842604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846092+0000","last_change":"2024-05-27T00:40:30.855023+0000","last_active":"2024-05-27T00:40:31.846092+0000","last_peered":"2024-05-27T00:40:31.846092+0000","last_clean":"2024-05-27T00:40:31.846092+0000","last_became_active":"2024-05-27T00:40:30.854711+0000","last_became_peered":"2024-05-27T00:40:30.854711+0000","last_unstale":"2024-05-27T00:40:31.846092+0000","last_undegraded":"2024-05-27T00:40:31.846092+0000","last_fullsized":"2024-05-27T00:40:31.846092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:09:09.478467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844453+0000","last_change":"2024-05-27T00:40:29.839332+0000","last_active":"2024-05-27T00:40:31.844453+0000","last_peered":"2024-05-27T00:40:31.844453+0000","last_clean":"2024-05-27T00:40:31.844453+0000","last_became_active":"2024-05-27T00:40:29.839058+0000","last_became_peered":"2024-05-27T00:40:29.839058+0000","last_unstale":"2024-05-27T00:40:31.844453+0000","last_undegraded":"2024-05-27T00:40:31.844453+0000","last_fullsized":"2024-05-27T00:40:31.844453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755413+0000","last_change":"2024-05-27T00:40:30.842022+0000","last_active":"2024-05-27T00:40:33.755413+0000","last_peered":"2024-05-27T00:40:33.755413+0000","last_clean":"2024-05-27T00:40:33.755413+0000","last_became_active":"2024-05-27T00:40:30.841565+0000","last_became_peered":"2024-05-27T00:40:30.841565+0000","last_unstale":"2024-05-27T00:40:33.755413+0000","last_undegraded":"2024-05-27T00:40:33.755413+0000","last_fullsized":"2024-05-27T00:40:33.755413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846503+0000","last_change":"2024-05-27T00:40:29.837660+0000","last_active":"2024-05-27T00:40:31.846503+0000","last_peered":"2024-05-27T00:40:31.846503+0000","last_clean":"2024-05-27T00:40:31.846503+0000","last_became_active":"2024-05-27T00:40:29.836617+0000","last_became_peered":"2024-05-27T00:40:29.836617+0000","last_unstale":"2024-05-27T00:40:31.846503+0000","last_undegraded":"2024-05-27T00:40:31.846503+0000","last_fullsized":"2024-05-27T00:40:31.846503+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:33.427371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846221+0000","last_change":"2024-05-27T00:40:30.852888+0000","last_active":"2024-05-27T00:40:31.846221+0000","last_peered":"2024-05-27T00:40:31.846221+0000","last_clean":"2024-05-27T00:40:31.846221+0000","last_became_active":"2024-05-27T00:40:30.852614+0000","last_became_peered":"2024-05-27T00:40:30.852614+0000","last_unstale":"2024-05-27T00:40:31.846221+0000","last_undegraded":"2024-05-27T00:40:31.846221+0000","last_fullsized":"2024-05-27T00:40:31.846221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:33:24.367596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755327+0000","last_change":"2024-05-27T00:40:29.844350+0000","last_active":"2024-05-27T00:40:33.755327+0000","last_peered":"2024-05-27T00:40:33.755327+0000","last_clean":"2024-05-27T00:40:33.755327+0000","last_became_active":"2024-05-27T00:40:29.843988+0000","last_became_peered":"2024-05-27T00:40:29.843988+0000","last_unstale":"2024-05-27T00:40:33.755327+0000","last_undegraded":"2024-05-27T00:40:33.755327+0000","last_fullsized":"2024-05-27T00:40:33.755327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846288+0000","last_change":"2024-05-27T00:40:30.860095+0000","last_active":"2024-05-27T00:40:31.846288+0000","last_peered":"2024-05-27T00:40:31.846288+0000","last_clean":"2024-05-27T00:40:31.846288+0000","last_became_active":"2024-05-27T00:40:30.858753+0000","last_became_peered":"2024-05-27T00:40:30.858753+0000","last_unstale":"2024-05-27T00:40:31.846288+0000","last_undegraded":"2024-05-27T00:40:31.846288+0000","last_fullsized":"2024-05-27T00:40:31.846288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:12.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963918+0000","last_change":"2024-05-27T00:40:29.836227+0000","last_active":"2024-05-27T00:40:31.963918+0000","last_peered":"2024-05-27T00:40:31.963918+0000","last_clean":"2024-05-27T00:40:31.963918+0000","last_became_active":"2024-05-27T00:40:29.835837+0000","last_became_peered":"2024-05-27T00:40:29.835837+0000","last_unstale":"2024-05-27T00:40:31.963918+0000","last_undegraded":"2024-05-27T00:40:31.963918+0000","last_fullsized":"2024-05-27T00:40:31.963918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:25.290676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963969+0000","last_change":"2024-05-27T00:40:30.848394+0000","last_active":"2024-05-27T00:40:31.963969+0000","last_peered":"2024-05-27T00:40:31.963969+0000","last_clean":"2024-05-27T00:40:31.963969+0000","last_became_active":"2024-05-27T00:40:30.846482+0000","last_became_peered":"2024-05-27T00:40:30.846482+0000","last_unstale":"2024-05-27T00:40:31.963969+0000","last_undegraded":"2024-05-27T00:40:31.963969+0000","last_fullsized":"2024-05-27T00:40:31.963969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:12.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756289+0000","last_change":"2024-05-27T00:40:29.838407+0000","last_active":"2024-05-27T00:40:33.756289+0000","last_peered":"2024-05-27T00:40:33.756289+0000","last_clean":"2024-05-27T00:40:33.756289+0000","last_became_active":"2024-05-27T00:40:29.837279+0000","last_became_peered":"2024-05-27T00:40:29.837279+0000","last_unstale":"2024-05-27T00:40:33.756289+0000","last_undegraded":"2024-05-27T00:40:33.756289+0000","last_fullsized":"2024-05-27T00:40:33.756289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:05.797447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756221+0000","last_change":"2024-05-27T00:40:30.846070+0000","last_active":"2024-05-27T00:40:33.756221+0000","last_peered":"2024-05-27T00:40:33.756221+0000","last_clean":"2024-05-27T00:40:33.756221+0000","last_became_active":"2024-05-27T00:40:30.845813+0000","last_became_peered":"2024-05-27T00:40:30.845813+0000","last_unstale":"2024-05-27T00:40:33.756221+0000","last_undegraded":"2024-05-27T00:40:33.756221+0000","last_fullsized":"2024-05-27T00:40:33.756221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755160+0000","last_change":"2024-05-27T00:40:29.836358+0000","last_active":"2024-05-27T00:40:33.755160+0000","last_peered":"2024-05-27T00:40:33.755160+0000","last_clean":"2024-05-27T00:40:33.755160+0000","last_became_active":"2024-05-27T00:40:29.836030+0000","last_became_peered":"2024-05-27T00:40:29.836030+0000","last_unstale":"2024-05-27T00:40:33.755160+0000","last_undegraded":"2024-05-27T00:40:33.755160+0000","last_fullsized":"2024-05-27T00:40:33.755160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:31.410996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754925+0000","last_change":"2024-05-27T00:40:30.858900+0000","last_active":"2024-05-27T00:40:33.754925+0000","last_peered":"2024-05-27T00:40:33.754925+0000","last_clean":"2024-05-27T00:40:33.754925+0000","last_became_active":"2024-05-27T00:40:30.858588+0000","last_became_peered":"2024-05-27T00:40:30.858588+0000","last_unstale":"2024-05-27T00:40:33.754925+0000","last_undegraded":"2024-05-27T00:40:33.754925+0000","last_fullsized":"2024-05-27T00:40:33.754925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:36.744482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844468+0000","last_change":"2024-05-27T00:40:29.843381+0000","last_active":"2024-05-27T00:40:31.844468+0000","last_peered":"2024-05-27T00:40:31.844468+0000","last_clean":"2024-05-27T00:40:31.844468+0000","last_became_active":"2024-05-27T00:40:29.843087+0000","last_became_peered":"2024-05-27T00:40:29.843087+0000","last_unstale":"2024-05-27T00:40:31.844468+0000","last_undegraded":"2024-05-27T00:40:31.844468+0000","last_fullsized":"2024-05-27T00:40:31.844468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:48.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755991+0000","last_change":"2024-05-27T00:40:30.844974+0000","last_active":"2024-05-27T00:40:33.755991+0000","last_peered":"2024-05-27T00:40:33.755991+0000","last_clean":"2024-05-27T00:40:33.755991+0000","last_became_active":"2024-05-27T00:40:30.843119+0000","last_became_peered":"2024-05-27T00:40:30.843119+0000","last_unstale":"2024-05-27T00:40:33.755991+0000","last_undegraded":"2024-05-27T00:40:33.755991+0000","last_fullsized":"2024-05-27T00:40:33.755991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754605+0000","last_change":"2024-05-27T00:40:29.831709+0000","last_active":"2024-05-27T00:40:33.754605+0000","last_peered":"2024-05-27T00:40:33.754605+0000","last_clean":"2024-05-27T00:40:33.754605+0000","last_became_active":"2024-05-27T00:40:29.831355+0000","last_became_peered":"2024-05-27T00:40:29.831355+0000","last_unstale":"2024-05-27T00:40:33.754605+0000","last_undegraded":"2024-05-27T00:40:33.754605+0000","last_fullsized":"2024-05-27T00:40:33.754605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:12.195311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754862+0000","last_change":"2024-05-27T00:40:30.844565+0000","last_active":"2024-05-27T00:40:33.754862+0000","last_peered":"2024-05-27T00:40:33.754862+0000","last_clean":"2024-05-27T00:40:33.754862+0000","last_became_active":"2024-05-27T00:40:30.844279+0000","last_became_peered":"2024-05-27T00:40:30.844279+0000","last_unstale":"2024-05-27T00:40:33.754862+0000","last_undegraded":"2024-05-27T00:40:33.754862+0000","last_fullsized":"2024-05-27T00:40:33.754862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:12.844381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754669+0000","last_change":"2024-05-27T00:40:29.835427+0000","last_active":"2024-05-27T00:40:33.754669+0000","last_peered":"2024-05-27T00:40:33.754669+0000","last_clean":"2024-05-27T00:40:33.754669+0000","last_became_active":"2024-05-27T00:40:29.834727+0000","last_became_peered":"2024-05-27T00:40:29.834727+0000","last_unstale":"2024-05-27T00:40:33.754669+0000","last_undegraded":"2024-05-27T00:40:33.754669+0000","last_fullsized":"2024-05-27T00:40:33.754669+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:58:42.956086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963716+0000","last_change":"2024-05-27T00:40:30.851594+0000","last_active":"2024-05-27T00:40:31.963716+0000","last_peered":"2024-05-27T00:40:31.963716+0000","last_clean":"2024-05-27T00:40:31.963716+0000","last_became_active":"2024-05-27T00:40:30.851318+0000","last_became_peered":"2024-05-27T00:40:30.851318+0000","last_unstale":"2024-05-27T00:40:31.963716+0000","last_undegraded":"2024-05-27T00:40:31.963716+0000","last_fullsized":"2024-05-27T00:40:31.963716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:23:13.579222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755911+0000","last_change":"2024-05-27T00:40:29.843742+0000","last_active":"2024-05-27T00:40:33.755911+0000","last_peered":"2024-05-27T00:40:33.755911+0000","last_clean":"2024-05-27T00:40:33.755911+0000","last_became_active":"2024-05-27T00:40:29.843445+0000","last_became_peered":"2024-05-27T00:40:29.843445+0000","last_unstale":"2024-05-27T00:40:33.755911+0000","last_undegraded":"2024-05-27T00:40:33.755911+0000","last_fullsized":"2024-05-27T00:40:33.755911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846350+0000","last_change":"2024-05-27T00:40:30.854867+0000","last_active":"2024-05-27T00:40:31.846350+0000","last_peered":"2024-05-27T00:40:31.846350+0000","last_clean":"2024-05-27T00:40:31.846350+0000","last_became_active":"2024-05-27T00:40:30.854579+0000","last_became_peered":"2024-05-27T00:40:30.854579+0000","last_unstale":"2024-05-27T00:40:31.846350+0000","last_undegraded":"2024-05-27T00:40:31.846350+0000","last_fullsized":"2024-05-27T00:40:31.846350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:06:27.438392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754435+0000","last_change":"2024-05-27T00:40:29.830333+0000","last_active":"2024-05-27T00:40:33.754435+0000","last_peered":"2024-05-27T00:40:33.754435+0000","last_clean":"2024-05-27T00:40:33.754435+0000","last_became_active":"2024-05-27T00:40:29.830050+0000","last_became_peered":"2024-05-27T00:40:29.830050+0000","last_unstale":"2024-05-27T00:40:33.754435+0000","last_undegraded":"2024-05-27T00:40:33.754435+0000","last_fullsized":"2024-05-27T00:40:33.754435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:01.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844646+0000","last_change":"2024-05-27T00:40:30.851081+0000","last_active":"2024-05-27T00:40:31.844646+0000","last_peered":"2024-05-27T00:40:31.844646+0000","last_clean":"2024-05-27T00:40:31.844646+0000","last_became_active":"2024-05-27T00:40:30.850755+0000","last_became_peered":"2024-05-27T00:40:30.850755+0000","last_unstale":"2024-05-27T00:40:31.844646+0000","last_undegraded":"2024-05-27T00:40:31.844646+0000","last_fullsized":"2024-05-27T00:40:31.844646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964070+0000","last_change":"2024-05-27T00:40:29.835772+0000","last_active":"2024-05-27T00:40:31.964070+0000","last_peered":"2024-05-27T00:40:31.964070+0000","last_clean":"2024-05-27T00:40:31.964070+0000","last_became_active":"2024-05-27T00:40:29.835453+0000","last_became_peered":"2024-05-27T00:40:29.835453+0000","last_unstale":"2024-05-27T00:40:31.964070+0000","last_undegraded":"2024-05-27T00:40:31.964070+0000","last_fullsized":"2024-05-27T00:40:31.964070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:05.797447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756274+0000","last_change":"2024-05-27T00:40:30.844109+0000","last_active":"2024-05-27T00:40:33.756274+0000","last_peered":"2024-05-27T00:40:33.756274+0000","last_clean":"2024-05-27T00:40:33.756274+0000","last_became_active":"2024-05-27T00:40:30.842384+0000","last_became_peered":"2024-05-27T00:40:30.842384+0000","last_unstale":"2024-05-27T00:40:33.756274+0000","last_undegraded":"2024-05-27T00:40:33.756274+0000","last_fullsized":"2024-05-27T00:40:33.756274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:51.877600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755046+0000","last_change":"2024-05-27T00:40:29.827858+0000","last_active":"2024-05-27T00:40:33.755046+0000","last_peered":"2024-05-27T00:40:33.755046+0000","last_clean":"2024-05-27T00:40:33.755046+0000","last_became_active":"2024-05-27T00:40:29.827589+0000","last_became_peered":"2024-05-27T00:40:29.827589+0000","last_unstale":"2024-05-27T00:40:33.755046+0000","last_undegraded":"2024-05-27T00:40:33.755046+0000","last_fullsized":"2024-05-27T00:40:33.755046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:28.730539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846318+0000","last_change":"2024-05-27T00:40:30.860768+0000","last_active":"2024-05-27T00:40:31.846318+0000","last_peered":"2024-05-27T00:40:31.846318+0000","last_clean":"2024-05-27T00:40:31.846318+0000","last_became_active":"2024-05-27T00:40:30.860451+0000","last_became_peered":"2024-05-27T00:40:30.860451+0000","last_unstale":"2024-05-27T00:40:31.846318+0000","last_undegraded":"2024-05-27T00:40:31.846318+0000","last_fullsized":"2024-05-27T00:40:31.846318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:43:23.638293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755169+0000","last_change":"2024-05-27T00:40:29.835183+0000","last_active":"2024-05-27T00:40:33.755169+0000","last_peered":"2024-05-27T00:40:33.755169+0000","last_clean":"2024-05-27T00:40:33.755169+0000","last_became_active":"2024-05-27T00:40:29.834922+0000","last_became_peered":"2024-05-27T00:40:29.834922+0000","last_unstale":"2024-05-27T00:40:33.755169+0000","last_undegraded":"2024-05-27T00:40:33.755169+0000","last_fullsized":"2024-05-27T00:40:33.755169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:28.086108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754715+0000","last_change":"2024-05-27T00:40:30.859747+0000","last_active":"2024-05-27T00:40:33.754715+0000","last_peered":"2024-05-27T00:40:33.754715+0000","last_clean":"2024-05-27T00:40:33.754715+0000","last_became_active":"2024-05-27T00:40:30.859488+0000","last_became_peered":"2024-05-27T00:40:30.859488+0000","last_unstale":"2024-05-27T00:40:33.754715+0000","last_undegraded":"2024-05-27T00:40:33.754715+0000","last_fullsized":"2024-05-27T00:40:33.754715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:51.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754941+0000","last_change":"2024-05-27T00:40:29.837690+0000","last_active":"2024-05-27T00:40:33.754941+0000","last_peered":"2024-05-27T00:40:33.754941+0000","last_clean":"2024-05-27T00:40:33.754941+0000","last_became_active":"2024-05-27T00:40:29.837411+0000","last_became_peered":"2024-05-27T00:40:29.837411+0000","last_unstale":"2024-05-27T00:40:33.754941+0000","last_undegraded":"2024-05-27T00:40:33.754941+0000","last_fullsized":"2024-05-27T00:40:33.754941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844355+0000","last_change":"2024-05-27T00:40:30.849204+0000","last_active":"2024-05-27T00:40:31.844355+0000","last_peered":"2024-05-27T00:40:31.844355+0000","last_clean":"2024-05-27T00:40:31.844355+0000","last_became_active":"2024-05-27T00:40:30.848866+0000","last_became_peered":"2024-05-27T00:40:30.848866+0000","last_unstale":"2024-05-27T00:40:31.844355+0000","last_undegraded":"2024-05-27T00:40:31.844355+0000","last_fullsized":"2024-05-27T00:40:31.844355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:56.389913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846650+0000","last_change":"2024-05-27T00:40:29.833805+0000","last_active":"2024-05-27T00:40:31.846650+0000","last_peered":"2024-05-27T00:40:31.846650+0000","last_clean":"2024-05-27T00:40:31.846650+0000","last_became_active":"2024-05-27T00:40:29.833515+0000","last_became_peered":"2024-05-27T00:40:29.833515+0000","last_unstale":"2024-05-27T00:40:31.846650+0000","last_undegraded":"2024-05-27T00:40:31.846650+0000","last_fullsized":"2024-05-27T00:40:31.846650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:05.002271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755459+0000","last_change":"2024-05-27T00:40:30.854612+0000","last_active":"2024-05-27T00:40:33.755459+0000","last_peered":"2024-05-27T00:40:33.755459+0000","last_clean":"2024-05-27T00:40:33.755459+0000","last_became_active":"2024-05-27T00:40:30.854334+0000","last_became_peered":"2024-05-27T00:40:30.854334+0000","last_unstale":"2024-05-27T00:40:33.755459+0000","last_undegraded":"2024-05-27T00:40:33.755459+0000","last_fullsized":"2024-05-27T00:40:33.755459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755280+0000","last_change":"2024-05-27T00:40:29.834692+0000","last_active":"2024-05-27T00:40:33.755280+0000","last_peered":"2024-05-27T00:40:33.755280+0000","last_clean":"2024-05-27T00:40:33.755280+0000","last_became_active":"2024-05-27T00:40:29.834412+0000","last_became_peered":"2024-05-27T00:40:29.834412+0000","last_unstale":"2024-05-27T00:40:33.755280+0000","last_undegraded":"2024-05-27T00:40:33.755280+0000","last_fullsized":"2024-05-27T00:40:33.755280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756089+0000","last_change":"2024-05-27T00:40:30.845502+0000","last_active":"2024-05-27T00:40:33.756089+0000","last_peered":"2024-05-27T00:40:33.756089+0000","last_clean":"2024-05-27T00:40:33.756089+0000","last_became_active":"2024-05-27T00:40:30.843028+0000","last_became_peered":"2024-05-27T00:40:30.843028+0000","last_unstale":"2024-05-27T00:40:33.756089+0000","last_undegraded":"2024-05-27T00:40:33.756089+0000","last_fullsized":"2024-05-27T00:40:33.756089+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846039+0000","last_change":"2024-05-27T00:40:29.835197+0000","last_active":"2024-05-27T00:40:31.846039+0000","last_peered":"2024-05-27T00:40:31.846039+0000","last_clean":"2024-05-27T00:40:31.846039+0000","last_became_active":"2024-05-27T00:40:29.834907+0000","last_became_peered":"2024-05-27T00:40:29.834907+0000","last_unstale":"2024-05-27T00:40:31.846039+0000","last_undegraded":"2024-05-27T00:40:31.846039+0000","last_fullsized":"2024-05-27T00:40:31.846039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:36.781562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963924+0000","last_change":"2024-05-27T00:40:30.859603+0000","last_active":"2024-05-27T00:40:31.963924+0000","last_peered":"2024-05-27T00:40:31.963924+0000","last_clean":"2024-05-27T00:40:31.963924+0000","last_became_active":"2024-05-27T00:40:30.859333+0000","last_became_peered":"2024-05-27T00:40:30.859333+0000","last_unstale":"2024-05-27T00:40:31.963924+0000","last_undegraded":"2024-05-27T00:40:31.963924+0000","last_fullsized":"2024-05-27T00:40:31.963924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:16.405801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756215+0000","last_change":"2024-05-27T00:40:29.843470+0000","last_active":"2024-05-27T00:40:33.756215+0000","last_peered":"2024-05-27T00:40:33.756215+0000","last_clean":"2024-05-27T00:40:33.756215+0000","last_became_active":"2024-05-27T00:40:29.843159+0000","last_became_peered":"2024-05-27T00:40:29.843159+0000","last_unstale":"2024-05-27T00:40:33.756215+0000","last_undegraded":"2024-05-27T00:40:33.756215+0000","last_fullsized":"2024-05-27T00:40:33.756215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:48.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755746+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:40:33.755746+0000","last_peered":"2024-05-27T00:40:33.755746+0000","last_clean":"2024-05-27T00:40:33.755746+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:40:33.755746+0000","last_undegraded":"2024-05-27T00:40:33.755746+0000","last_fullsized":"2024-05-27T00:40:33.755746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":197909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":197439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":198973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":197417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":781037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:40:38.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T00:40:38.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.864+0000 7f51095a9640 1 Processor -- start 2024-05-27T00:40:38.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.864+0000 7f51095a9640 1 -- start start 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51095a9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 0x7f510415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51095a9640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5104162430 0x7f51040af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51095a9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51040afc30 con 0x7f51040af7a0 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51095a9640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f51040b4630 con 0x7f5104162430 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51095a9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f51040b3a20 con 0x7f5104162050 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51037fe640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f51040af7a0 0x7f51040b4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58994/0 (socket says 172.21.15.111:58994) 2024-05-27T00:40:38.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.865+0000 7f51037fe640 1 -- 172.21.15.111:0/2546538446 learned_addr learned my addr 172.21.15.111:0/2546538446 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f5102ffd640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 0x7f510415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f50fbfff640 1 --2- 172.21.15.111:0/2546538446 >> v2:172.21.15.153:3300/0 conn(0x7f5104162430 0x7f51040af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803991943 0 0) 0x7f51040afc30 con 0x7f51040af7a0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f0003520 con 0x7f51040af7a0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 72239817 0 0) 0x7f50f0003520 con 0x7f51040af7a0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 msgr2=0x7f510415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 0x7f510415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 >> v2:172.21.15.153:3300/0 conn(0x7f5104162430 msgr2=0x7f51040af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 --2- 172.21.15.111:0/2546538446 >> v2:172.21.15.153:3300/0 conn(0x7f5104162430 0x7f51040af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040b3d00 con 0x7f51040af7a0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f50fc0019f0 con 0x7f51040af7a0 2024-05-27T00:40:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 >> v1:172.21.15.111:6789/0 conn(0x7f51040af7a0 legacy=0x7f51040b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50f00039b0 0x7f50f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 0x7f50f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f0004d40 0x7f50f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f51040b4630 con 0x7f50f00039b0 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f51040b3a20 con 0x7f50f0004d40 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f50f000b9c0 con 0x7f5104162050 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f5102ffd640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50f00039b0 0x7f50f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f50fbfff640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 0x7f50f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f51037fe640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f0004d40 0x7f50f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f5102ffd640 1 -- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 msgr2=0x7f50f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.866+0000 7f5102ffd640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f5104162050 0x7f50f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f5102ffd640 1 -- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f0004d40 msgr2=0x7f50f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f5102ffd640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f0004d40 0x7f50f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f5102ffd640 1 -- 172.21.15.111:0/2546538446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f0003520 con 0x7f50f00039b0 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f51037fe640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f0004d40 0x7f50f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f5102ffd640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50f00039b0 0x7f50f0003dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009ac0 tx=0x7f50f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ffe20960a5a14c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4036070 con 0x7f50f00039b0 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f4031400 con 0x7f50f00039b0 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.867+0000 7f51027fc640 1 -- 172.21.15.111:0/2546538446 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4009690 con 0x7f50f00039b0 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51095a9640 1 -- 172.21.15.111:0/2546538446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50f00039b0 msgr2=0x7f50f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51095a9640 1 --2- 172.21.15.111:0/2546538446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50f00039b0 0x7f50f0003dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009ac0 tx=0x7f50f0005830 comp rx=0 tx=0).stop 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51037fe640 1 -- 172.21.15.111:0/2546538446 reap_dead start 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51095a9640 1 -- 172.21.15.111:0/2546538446 shutdown_connections 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51095a9640 1 -- 172.21.15.111:0/2546538446 >> 172.21.15.111:0/2546538446 conn(0x7f5104072a30 msgr2=0x7f510406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51095a9640 1 -- 172.21.15.111:0/2546538446 shutdown_connections 2024-05-27T00:40:38.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.868+0000 7f51095a9640 1 -- 172.21.15.111:0/2546538446 wait complete. 2024-05-27T00:40:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.869+0000 7f51095a9640 1 Processor -- start 2024-05-27T00:40:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.869+0000 7f51095a9640 1 -- start start 2024-05-27T00:40:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51095a9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f00039b0 0x7f51040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51095a9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 0x7f51040cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51095a9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51041e8580 0x7f51041e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51095a9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f510415f9b0 con 0x7f51040cd4e0 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51095a9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f51040b42c0 con 0x7f50f00039b0 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51095a9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f510415fb30 con 0x7f51041e8580 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 0x7f51040cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 0x7f51040cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59550/0 (socket says 172.21.15.111:59550) 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 -- 172.21.15.111:0/1155757637 learned_addr learned my addr 172.21.15.111:0/1155757637 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f51037fe640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51041e8580 0x7f51041e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f5102ffd640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f00039b0 0x7f51040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 -- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51041e8580 msgr2=0x7f51041e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51041e8580 0x7f51041e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 -- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f00039b0 msgr2=0x7f51040ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f00039b0 0x7f51040ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f50fbfff640 1 -- 172.21.15.111:0/1155757637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51041e8f40 con 0x7f51040cd4e0 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.870+0000 7f5102ffd640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f00039b0 0x7f51040ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:38.876 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.871+0000 7f50fbfff640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 0x7f51040cd960 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011aa0 tx=0x7f50ec011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.871+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00fb20 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.871+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec00fcc0 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.871+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51041e77c0 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.871+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec014660 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.871+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51041e7d80 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.872+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5104068680 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.872+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50ec012040 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.873+0000 7f50fb7fe640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50e406be60 0x7f50e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.873+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f50ec08b5d0 con 0x7f51040cd4e0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.873+0000 7f5102ffd640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50e406be60 0x7f50e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:38.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.874+0000 7f5102ffd640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50e406be60 0x7f50e406e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f50f4033000 tx=0x7f50f4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:38.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:38.877+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50ec019030 con 0x7f51040cd4e0 2024-05-27T00:40:39.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.049+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 --> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51040aef70 con 0x7f50e406be60 2024-05-27T00:40:39.056 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.056+0000 7f50fb7fe640 1 -- 172.21.15.111:0/1155757637 <== mgr.4102 v2:172.21.15.153:6832/2917284088 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f51040aef70 con 0x7f50e406be60 2024-05-27T00:40:39.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:39.060 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-05-27T00:40:39.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50e406be60 msgr2=0x7f50e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50e406be60 0x7f50e406e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f50f4033000 tx=0x7f50f4008030 comp rx=0 tx=0).stop 2024-05-27T00:40:39.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 msgr2=0x7f51040cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 0x7f51040cd960 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011aa0 tx=0x7f50ec011f60 comp rx=0 tx=0).stop 2024-05-27T00:40:39.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 shutdown_connections 2024-05-27T00:40:39.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f50e406be60 0x7f50e406e320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f51041e8580 0x7f51041e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040cd4e0 0x7f51040cd960 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 --2- 172.21.15.111:0/1155757637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50f00039b0 0x7f51040ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 >> 172.21.15.111:0/1155757637 conn(0x7f5104072a30 msgr2=0x7f510406e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:39.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 shutdown_connections 2024-05-27T00:40:39.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.059+0000 7f51095a9640 1 -- 172.21.15.111:0/1155757637 wait complete. 2024-05-27T00:40:39.070 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-05-27T00:40:38.944009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220888,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753832,"statfs":{"total":773094113280,"available":772867923968,"internally_reserved":0,"allocated":4820992,"data_stored":2744096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.113897"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846398+0000","last_change":"2024-05-27T00:40:30.855401+0000","last_active":"2024-05-27T00:40:31.846398+0000","last_peered":"2024-05-27T00:40:31.846398+0000","last_clean":"2024-05-27T00:40:31.846398+0000","last_became_active":"2024-05-27T00:40:30.852565+0000","last_became_peered":"2024-05-27T00:40:30.852565+0000","last_unstale":"2024-05-27T00:40:31.846398+0000","last_undegraded":"2024-05-27T00:40:31.846398+0000","last_fullsized":"2024-05-27T00:40:31.846398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:43.157586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846269+0000","last_change":"2024-05-27T00:40:30.855430+0000","last_active":"2024-05-27T00:40:31.846269+0000","last_peered":"2024-05-27T00:40:31.846269+0000","last_clean":"2024-05-27T00:40:31.846269+0000","last_became_active":"2024-05-27T00:40:30.855163+0000","last_became_peered":"2024-05-27T00:40:30.855163+0000","last_unstale":"2024-05-27T00:40:31.846269+0000","last_undegraded":"2024-05-27T00:40:31.846269+0000","last_fullsized":"2024-05-27T00:40:31.846269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:16.405801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755165+0000","last_change":"2024-05-27T00:40:30.854858+0000","last_active":"2024-05-27T00:40:33.755165+0000","last_peered":"2024-05-27T00:40:33.755165+0000","last_clean":"2024-05-27T00:40:33.755165+0000","last_became_active":"2024-05-27T00:40:30.854577+0000","last_became_peered":"2024-05-27T00:40:30.854577+0000","last_unstale":"2024-05-27T00:40:33.755165+0000","last_undegraded":"2024-05-27T00:40:33.755165+0000","last_fullsized":"2024-05-27T00:40:33.755165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755138+0000","last_change":"2024-05-27T00:40:30.852335+0000","last_active":"2024-05-27T00:40:33.755138+0000","last_peered":"2024-05-27T00:40:33.755138+0000","last_clean":"2024-05-27T00:40:33.755138+0000","last_became_active":"2024-05-27T00:40:30.852014+0000","last_became_peered":"2024-05-27T00:40:30.852014+0000","last_unstale":"2024-05-27T00:40:33.755138+0000","last_undegraded":"2024-05-27T00:40:33.755138+0000","last_fullsized":"2024-05-27T00:40:33.755138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:29:13.936412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754549+0000","last_change":"2024-05-27T00:40:30.847265+0000","last_active":"2024-05-27T00:40:33.754549+0000","last_peered":"2024-05-27T00:40:33.754549+0000","last_clean":"2024-05-27T00:40:33.754549+0000","last_became_active":"2024-05-27T00:40:30.846994+0000","last_became_peered":"2024-05-27T00:40:30.846994+0000","last_unstale":"2024-05-27T00:40:33.754549+0000","last_undegraded":"2024-05-27T00:40:33.754549+0000","last_fullsized":"2024-05-27T00:40:33.754549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:50:40.605562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844389+0000","last_change":"2024-05-27T00:40:30.854665+0000","last_active":"2024-05-27T00:40:31.844389+0000","last_peered":"2024-05-27T00:40:31.844389+0000","last_clean":"2024-05-27T00:40:31.844389+0000","last_became_active":"2024-05-27T00:40:30.854381+0000","last_became_peered":"2024-05-27T00:40:30.854381+0000","last_unstale":"2024-05-27T00:40:31.844389+0000","last_undegraded":"2024-05-27T00:40:31.844389+0000","last_fullsized":"2024-05-27T00:40:31.844389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:51.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963523+0000","last_change":"2024-05-27T00:40:30.853683+0000","last_active":"2024-05-27T00:40:31.963523+0000","last_peered":"2024-05-27T00:40:31.963523+0000","last_clean":"2024-05-27T00:40:31.963523+0000","last_became_active":"2024-05-27T00:40:30.853441+0000","last_became_peered":"2024-05-27T00:40:30.853441+0000","last_unstale":"2024-05-27T00:40:31.963523+0000","last_undegraded":"2024-05-27T00:40:31.963523+0000","last_fullsized":"2024-05-27T00:40:31.963523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846007+0000","last_change":"2024-05-27T00:40:30.855511+0000","last_active":"2024-05-27T00:40:31.846007+0000","last_peered":"2024-05-27T00:40:31.846007+0000","last_clean":"2024-05-27T00:40:31.846007+0000","last_became_active":"2024-05-27T00:40:30.854581+0000","last_became_peered":"2024-05-27T00:40:30.854581+0000","last_unstale":"2024-05-27T00:40:31.846007+0000","last_undegraded":"2024-05-27T00:40:31.846007+0000","last_fullsized":"2024-05-27T00:40:31.846007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:23:13.579222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755351+0000","last_change":"2024-05-27T00:40:30.854970+0000","last_active":"2024-05-27T00:40:33.755351+0000","last_peered":"2024-05-27T00:40:33.755351+0000","last_clean":"2024-05-27T00:40:33.755351+0000","last_became_active":"2024-05-27T00:40:30.854622+0000","last_became_peered":"2024-05-27T00:40:30.854622+0000","last_unstale":"2024-05-27T00:40:33.755351+0000","last_undegraded":"2024-05-27T00:40:33.755351+0000","last_fullsized":"2024-05-27T00:40:33.755351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963840+0000","last_change":"2024-05-27T00:40:30.842451+0000","last_active":"2024-05-27T00:40:31.963840+0000","last_peered":"2024-05-27T00:40:31.963840+0000","last_clean":"2024-05-27T00:40:31.963840+0000","last_became_active":"2024-05-27T00:40:30.841621+0000","last_became_peered":"2024-05-27T00:40:30.841621+0000","last_unstale":"2024-05-27T00:40:31.963840+0000","last_undegraded":"2024-05-27T00:40:31.963840+0000","last_fullsized":"2024-05-27T00:40:31.963840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:33:24.367596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964105+0000","last_change":"2024-05-27T00:40:30.847791+0000","last_active":"2024-05-27T00:40:31.964105+0000","last_peered":"2024-05-27T00:40:31.964105+0000","last_clean":"2024-05-27T00:40:31.964105+0000","last_became_active":"2024-05-27T00:40:30.847451+0000","last_became_peered":"2024-05-27T00:40:30.847451+0000","last_unstale":"2024-05-27T00:40:31.964105+0000","last_undegraded":"2024-05-27T00:40:31.964105+0000","last_fullsized":"2024-05-27T00:40:31.964105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:43:23.638293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755117+0000","last_change":"2024-05-27T00:40:30.842233+0000","last_active":"2024-05-27T00:40:33.755117+0000","last_peered":"2024-05-27T00:40:33.755117+0000","last_clean":"2024-05-27T00:40:33.755117+0000","last_became_active":"2024-05-27T00:40:30.841968+0000","last_became_peered":"2024-05-27T00:40:30.841968+0000","last_unstale":"2024-05-27T00:40:33.755117+0000","last_undegraded":"2024-05-27T00:40:33.755117+0000","last_fullsized":"2024-05-27T00:40:33.755117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963532+0000","last_change":"2024-05-27T00:40:30.842018+0000","last_active":"2024-05-27T00:40:31.963532+0000","last_peered":"2024-05-27T00:40:31.963532+0000","last_clean":"2024-05-27T00:40:31.963532+0000","last_became_active":"2024-05-27T00:40:30.841715+0000","last_became_peered":"2024-05-27T00:40:30.841715+0000","last_unstale":"2024-05-27T00:40:31.963532+0000","last_undegraded":"2024-05-27T00:40:31.963532+0000","last_fullsized":"2024-05-27T00:40:31.963532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:43:04.866508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755418+0000","last_change":"2024-05-27T00:40:30.856740+0000","last_active":"2024-05-27T00:40:33.755418+0000","last_peered":"2024-05-27T00:40:33.755418+0000","last_clean":"2024-05-27T00:40:33.755418+0000","last_became_active":"2024-05-27T00:40:30.856474+0000","last_became_peered":"2024-05-27T00:40:30.856474+0000","last_unstale":"2024-05-27T00:40:33.755418+0000","last_undegraded":"2024-05-27T00:40:33.755418+0000","last_fullsized":"2024-05-27T00:40:33.755418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963508+0000","last_change":"2024-05-27T00:40:30.845060+0000","last_active":"2024-05-27T00:40:31.963508+0000","last_peered":"2024-05-27T00:40:31.963508+0000","last_clean":"2024-05-27T00:40:31.963508+0000","last_became_active":"2024-05-27T00:40:30.844505+0000","last_became_peered":"2024-05-27T00:40:30.844505+0000","last_unstale":"2024-05-27T00:40:31.963508+0000","last_undegraded":"2024-05-27T00:40:31.963508+0000","last_fullsized":"2024-05-27T00:40:31.963508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755545+0000","last_change":"2024-05-27T00:40:30.851915+0000","last_active":"2024-05-27T00:40:33.755545+0000","last_peered":"2024-05-27T00:40:33.755545+0000","last_clean":"2024-05-27T00:40:33.755545+0000","last_became_active":"2024-05-27T00:40:30.851259+0000","last_became_peered":"2024-05-27T00:40:30.851259+0000","last_unstale":"2024-05-27T00:40:33.755545+0000","last_undegraded":"2024-05-27T00:40:33.755545+0000","last_fullsized":"2024-05-27T00:40:33.755545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755992+0000","last_change":"2024-05-27T00:40:30.851666+0000","last_active":"2024-05-27T00:40:33.755992+0000","last_peered":"2024-05-27T00:40:33.755992+0000","last_clean":"2024-05-27T00:40:33.755992+0000","last_became_active":"2024-05-27T00:40:30.851232+0000","last_became_peered":"2024-05-27T00:40:30.851232+0000","last_unstale":"2024-05-27T00:40:33.755992+0000","last_undegraded":"2024-05-27T00:40:33.755992+0000","last_fullsized":"2024-05-27T00:40:33.755992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:47:48.323260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847151+0000","last_change":"2024-05-27T00:40:30.848089+0000","last_active":"2024-05-27T00:40:31.847151+0000","last_peered":"2024-05-27T00:40:31.847151+0000","last_clean":"2024-05-27T00:40:31.847151+0000","last_became_active":"2024-05-27T00:40:30.847800+0000","last_became_peered":"2024-05-27T00:40:30.847800+0000","last_unstale":"2024-05-27T00:40:31.847151+0000","last_undegraded":"2024-05-27T00:40:31.847151+0000","last_fullsized":"2024-05-27T00:40:31.847151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:49:44.200574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755204+0000","last_change":"2024-05-27T00:40:30.845947+0000","last_active":"2024-05-27T00:40:33.755204+0000","last_peered":"2024-05-27T00:40:33.755204+0000","last_clean":"2024-05-27T00:40:33.755204+0000","last_became_active":"2024-05-27T00:40:30.845659+0000","last_became_peered":"2024-05-27T00:40:30.845659+0000","last_unstale":"2024-05-27T00:40:33.755204+0000","last_undegraded":"2024-05-27T00:40:33.755204+0000","last_fullsized":"2024-05-27T00:40:33.755204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:29.087885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846967+0000","last_change":"2024-05-27T00:40:30.853177+0000","last_active":"2024-05-27T00:40:31.846967+0000","last_peered":"2024-05-27T00:40:31.846967+0000","last_clean":"2024-05-27T00:40:31.846967+0000","last_became_active":"2024-05-27T00:40:30.852953+0000","last_became_peered":"2024-05-27T00:40:30.852953+0000","last_unstale":"2024-05-27T00:40:31.846967+0000","last_undegraded":"2024-05-27T00:40:31.846967+0000","last_fullsized":"2024-05-27T00:40:31.846967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:06.799224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963712+0000","last_change":"2024-05-27T00:40:30.844500+0000","last_active":"2024-05-27T00:40:31.963712+0000","last_peered":"2024-05-27T00:40:31.963712+0000","last_clean":"2024-05-27T00:40:31.963712+0000","last_became_active":"2024-05-27T00:40:30.844193+0000","last_became_peered":"2024-05-27T00:40:30.844193+0000","last_unstale":"2024-05-27T00:40:31.963712+0000","last_undegraded":"2024-05-27T00:40:31.963712+0000","last_fullsized":"2024-05-27T00:40:31.963712+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:51.877600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755742+0000","last_change":"2024-05-27T00:40:30.851942+0000","last_active":"2024-05-27T00:40:33.755742+0000","last_peered":"2024-05-27T00:40:33.755742+0000","last_clean":"2024-05-27T00:40:33.755742+0000","last_became_active":"2024-05-27T00:40:30.850621+0000","last_became_peered":"2024-05-27T00:40:30.850621+0000","last_unstale":"2024-05-27T00:40:33.755742+0000","last_undegraded":"2024-05-27T00:40:33.755742+0000","last_fullsized":"2024-05-27T00:40:33.755742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755649+0000","last_change":"2024-05-27T00:40:30.845599+0000","last_active":"2024-05-27T00:40:33.755649+0000","last_peered":"2024-05-27T00:40:33.755649+0000","last_clean":"2024-05-27T00:40:33.755649+0000","last_became_active":"2024-05-27T00:40:30.845119+0000","last_became_peered":"2024-05-27T00:40:30.845119+0000","last_unstale":"2024-05-27T00:40:33.755649+0000","last_undegraded":"2024-05-27T00:40:33.755649+0000","last_fullsized":"2024-05-27T00:40:33.755649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754550+0000","last_change":"2024-05-27T00:40:30.840661+0000","last_active":"2024-05-27T00:40:33.754550+0000","last_peered":"2024-05-27T00:40:33.754550+0000","last_clean":"2024-05-27T00:40:33.754550+0000","last_became_active":"2024-05-27T00:40:30.840388+0000","last_became_peered":"2024-05-27T00:40:30.840388+0000","last_unstale":"2024-05-27T00:40:33.754550+0000","last_undegraded":"2024-05-27T00:40:33.754550+0000","last_fullsized":"2024-05-27T00:40:33.754550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:06.004048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847108+0000","last_change":"2024-05-27T00:40:30.849378+0000","last_active":"2024-05-27T00:40:31.847108+0000","last_peered":"2024-05-27T00:40:31.847108+0000","last_clean":"2024-05-27T00:40:31.847108+0000","last_became_active":"2024-05-27T00:40:30.848917+0000","last_became_peered":"2024-05-27T00:40:30.848917+0000","last_unstale":"2024-05-27T00:40:31.847108+0000","last_undegraded":"2024-05-27T00:40:31.847108+0000","last_fullsized":"2024-05-27T00:40:31.847108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:56.990399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844659+0000","last_change":"2024-05-27T00:40:30.850361+0000","last_active":"2024-05-27T00:40:31.844659+0000","last_peered":"2024-05-27T00:40:31.844659+0000","last_clean":"2024-05-27T00:40:31.844659+0000","last_became_active":"2024-05-27T00:40:30.850080+0000","last_became_peered":"2024-05-27T00:40:30.850080+0000","last_unstale":"2024-05-27T00:40:31.844659+0000","last_undegraded":"2024-05-27T00:40:31.844659+0000","last_fullsized":"2024-05-27T00:40:31.844659+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:06.070182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847120+0000","last_change":"2024-05-27T00:40:30.842686+0000","last_active":"2024-05-27T00:40:31.847120+0000","last_peered":"2024-05-27T00:40:31.847120+0000","last_clean":"2024-05-27T00:40:31.847120+0000","last_became_active":"2024-05-27T00:40:30.842431+0000","last_became_peered":"2024-05-27T00:40:30.842431+0000","last_unstale":"2024-05-27T00:40:31.847120+0000","last_undegraded":"2024-05-27T00:40:31.847120+0000","last_fullsized":"2024-05-27T00:40:31.847120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:48.622372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963820+0000","last_change":"2024-05-27T00:40:30.856168+0000","last_active":"2024-05-27T00:40:31.963820+0000","last_peered":"2024-05-27T00:40:31.963820+0000","last_clean":"2024-05-27T00:40:31.963820+0000","last_became_active":"2024-05-27T00:40:30.855908+0000","last_became_peered":"2024-05-27T00:40:30.855908+0000","last_unstale":"2024-05-27T00:40:31.963820+0000","last_undegraded":"2024-05-27T00:40:31.963820+0000","last_fullsized":"2024-05-27T00:40:31.963820+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:37:28.064481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846932+0000","last_change":"2024-05-27T00:40:30.844268+0000","last_active":"2024-05-27T00:40:31.846932+0000","last_peered":"2024-05-27T00:40:31.846932+0000","last_clean":"2024-05-27T00:40:31.846932+0000","last_became_active":"2024-05-27T00:40:30.843982+0000","last_became_peered":"2024-05-27T00:40:30.843982+0000","last_unstale":"2024-05-27T00:40:31.846932+0000","last_undegraded":"2024-05-27T00:40:31.846932+0000","last_fullsized":"2024-05-27T00:40:31.846932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:03.839740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846947+0000","last_change":"2024-05-27T00:40:30.855497+0000","last_active":"2024-05-27T00:40:31.846947+0000","last_peered":"2024-05-27T00:40:31.846947+0000","last_clean":"2024-05-27T00:40:31.846947+0000","last_became_active":"2024-05-27T00:40:30.855231+0000","last_became_peered":"2024-05-27T00:40:30.855231+0000","last_unstale":"2024-05-27T00:40:31.846947+0000","last_undegraded":"2024-05-27T00:40:31.846947+0000","last_fullsized":"2024-05-27T00:40:31.846947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:26.292453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755889+0000","last_change":"2024-05-27T00:40:30.846422+0000","last_active":"2024-05-27T00:40:33.755889+0000","last_peered":"2024-05-27T00:40:33.755889+0000","last_clean":"2024-05-27T00:40:33.755889+0000","last_became_active":"2024-05-27T00:40:30.845272+0000","last_became_peered":"2024-05-27T00:40:30.845272+0000","last_unstale":"2024-05-27T00:40:33.755889+0000","last_undegraded":"2024-05-27T00:40:33.755889+0000","last_fullsized":"2024-05-27T00:40:33.755889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:08:49.450082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844307+0000","last_change":"2024-05-27T00:40:30.843789+0000","last_active":"2024-05-27T00:40:31.844307+0000","last_peered":"2024-05-27T00:40:31.844307+0000","last_clean":"2024-05-27T00:40:31.844307+0000","last_became_active":"2024-05-27T00:40:30.843473+0000","last_became_peered":"2024-05-27T00:40:30.843473+0000","last_unstale":"2024-05-27T00:40:31.844307+0000","last_undegraded":"2024-05-27T00:40:31.844307+0000","last_fullsized":"2024-05-27T00:40:31.844307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:37.783339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756208+0000","last_change":"2024-05-27T00:40:30.846361+0000","last_active":"2024-05-27T00:40:33.756208+0000","last_peered":"2024-05-27T00:40:33.756208+0000","last_clean":"2024-05-27T00:40:33.756208+0000","last_became_active":"2024-05-27T00:40:30.846099+0000","last_became_peered":"2024-05-27T00:40:30.846099+0000","last_unstale":"2024-05-27T00:40:33.756208+0000","last_undegraded":"2024-05-27T00:40:33.756208+0000","last_fullsized":"2024-05-27T00:40:33.756208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:43:04.866508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.847082+0000","last_change":"2024-05-27T00:40:29.834920+0000","last_active":"2024-05-27T00:40:31.847082+0000","last_peered":"2024-05-27T00:40:31.847082+0000","last_clean":"2024-05-27T00:40:31.847082+0000","last_became_active":"2024-05-27T00:40:29.834624+0000","last_became_peered":"2024-05-27T00:40:29.834624+0000","last_unstale":"2024-05-27T00:40:31.847082+0000","last_undegraded":"2024-05-27T00:40:31.847082+0000","last_fullsized":"2024-05-27T00:40:31.847082+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:50:39.603785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844620+0000","last_change":"2024-05-27T00:40:30.851854+0000","last_active":"2024-05-27T00:40:31.844620+0000","last_peered":"2024-05-27T00:40:31.844620+0000","last_clean":"2024-05-27T00:40:31.844620+0000","last_became_active":"2024-05-27T00:40:30.851553+0000","last_became_peered":"2024-05-27T00:40:30.851553+0000","last_unstale":"2024-05-27T00:40:31.844620+0000","last_undegraded":"2024-05-27T00:40:31.844620+0000","last_fullsized":"2024-05-27T00:40:31.844620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:12.844381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754975+0000","last_change":"2024-05-27T00:40:29.841137+0000","last_active":"2024-05-27T00:40:33.754975+0000","last_peered":"2024-05-27T00:40:33.754975+0000","last_clean":"2024-05-27T00:40:33.754975+0000","last_became_active":"2024-05-27T00:40:29.840779+0000","last_became_peered":"2024-05-27T00:40:29.840779+0000","last_unstale":"2024-05-27T00:40:33.754975+0000","last_undegraded":"2024-05-27T00:40:33.754975+0000","last_fullsized":"2024-05-27T00:40:33.754975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756351+0000","last_change":"2024-05-27T00:40:30.851783+0000","last_active":"2024-05-27T00:40:33.756351+0000","last_peered":"2024-05-27T00:40:33.756351+0000","last_clean":"2024-05-27T00:40:33.756351+0000","last_became_active":"2024-05-27T00:40:30.851513+0000","last_became_peered":"2024-05-27T00:40:30.851513+0000","last_unstale":"2024-05-27T00:40:33.756351+0000","last_undegraded":"2024-05-27T00:40:33.756351+0000","last_fullsized":"2024-05-27T00:40:33.756351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844676+0000","last_change":"2024-05-27T00:40:29.839689+0000","last_active":"2024-05-27T00:40:31.844676+0000","last_peered":"2024-05-27T00:40:31.844676+0000","last_clean":"2024-05-27T00:40:31.844676+0000","last_became_active":"2024-05-27T00:40:29.839397+0000","last_became_peered":"2024-05-27T00:40:29.839397+0000","last_unstale":"2024-05-27T00:40:31.844676+0000","last_undegraded":"2024-05-27T00:40:31.844676+0000","last_fullsized":"2024-05-27T00:40:31.844676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846352+0000","last_change":"2024-05-27T00:40:30.856094+0000","last_active":"2024-05-27T00:40:31.846352+0000","last_peered":"2024-05-27T00:40:31.846352+0000","last_clean":"2024-05-27T00:40:31.846352+0000","last_became_active":"2024-05-27T00:40:30.855830+0000","last_became_peered":"2024-05-27T00:40:30.855830+0000","last_unstale":"2024-05-27T00:40:31.846352+0000","last_undegraded":"2024-05-27T00:40:31.846352+0000","last_fullsized":"2024-05-27T00:40:31.846352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:20.414809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755950+0000","last_change":"2024-05-27T00:40:29.834650+0000","last_active":"2024-05-27T00:40:33.755950+0000","last_peered":"2024-05-27T00:40:33.755950+0000","last_clean":"2024-05-27T00:40:33.755950+0000","last_became_active":"2024-05-27T00:40:29.834368+0000","last_became_peered":"2024-05-27T00:40:29.834368+0000","last_unstale":"2024-05-27T00:40:33.755950+0000","last_undegraded":"2024-05-27T00:40:33.755950+0000","last_fullsized":"2024-05-27T00:40:33.755950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:55.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754551+0000","last_change":"2024-05-27T00:40:30.849028+0000","last_active":"2024-05-27T00:40:33.754551+0000","last_peered":"2024-05-27T00:40:33.754551+0000","last_clean":"2024-05-27T00:40:33.754551+0000","last_became_active":"2024-05-27T00:40:30.848705+0000","last_became_peered":"2024-05-27T00:40:30.848705+0000","last_unstale":"2024-05-27T00:40:33.754551+0000","last_undegraded":"2024-05-27T00:40:33.754551+0000","last_fullsized":"2024-05-27T00:40:33.754551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:37.783339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963613+0000","last_change":"2024-05-27T00:40:29.837136+0000","last_active":"2024-05-27T00:40:31.963613+0000","last_peered":"2024-05-27T00:40:31.963613+0000","last_clean":"2024-05-27T00:40:31.963613+0000","last_became_active":"2024-05-27T00:40:29.836334+0000","last_became_peered":"2024-05-27T00:40:29.836334+0000","last_unstale":"2024-05-27T00:40:31.963613+0000","last_undegraded":"2024-05-27T00:40:31.963613+0000","last_fullsized":"2024-05-27T00:40:31.963613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:55.988622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963773+0000","last_change":"2024-05-27T00:40:30.844814+0000","last_active":"2024-05-27T00:40:31.963773+0000","last_peered":"2024-05-27T00:40:31.963773+0000","last_clean":"2024-05-27T00:40:31.963773+0000","last_became_active":"2024-05-27T00:40:30.844140+0000","last_became_peered":"2024-05-27T00:40:30.844140+0000","last_unstale":"2024-05-27T00:40:31.963773+0000","last_undegraded":"2024-05-27T00:40:31.963773+0000","last_fullsized":"2024-05-27T00:40:31.963773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:09:09.478467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755478+0000","last_change":"2024-05-27T00:40:29.843655+0000","last_active":"2024-05-27T00:40:33.755478+0000","last_peered":"2024-05-27T00:40:33.755478+0000","last_clean":"2024-05-27T00:40:33.755478+0000","last_became_active":"2024-05-27T00:40:29.843287+0000","last_became_peered":"2024-05-27T00:40:29.843287+0000","last_unstale":"2024-05-27T00:40:33.755478+0000","last_undegraded":"2024-05-27T00:40:33.755478+0000","last_fullsized":"2024-05-27T00:40:33.755478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:28.086108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756101+0000","last_change":"2024-05-27T00:40:30.858629+0000","last_active":"2024-05-27T00:40:33.756101+0000","last_peered":"2024-05-27T00:40:33.756101+0000","last_clean":"2024-05-27T00:40:33.756101+0000","last_became_active":"2024-05-27T00:40:30.858341+0000","last_became_peered":"2024-05-27T00:40:30.858341+0000","last_unstale":"2024-05-27T00:40:33.756101+0000","last_undegraded":"2024-05-27T00:40:33.756101+0000","last_fullsized":"2024-05-27T00:40:33.756101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:08:49.450082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844334+0000","last_change":"2024-05-27T00:40:29.838981+0000","last_active":"2024-05-27T00:40:31.844334+0000","last_peered":"2024-05-27T00:40:31.844334+0000","last_clean":"2024-05-27T00:40:31.844334+0000","last_became_active":"2024-05-27T00:40:29.838673+0000","last_became_peered":"2024-05-27T00:40:29.838673+0000","last_unstale":"2024-05-27T00:40:31.844334+0000","last_undegraded":"2024-05-27T00:40:31.844334+0000","last_fullsized":"2024-05-27T00:40:31.844334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846176+0000","last_change":"2024-05-27T00:40:30.860388+0000","last_active":"2024-05-27T00:40:31.846176+0000","last_peered":"2024-05-27T00:40:31.846176+0000","last_clean":"2024-05-27T00:40:31.846176+0000","last_became_active":"2024-05-27T00:40:30.859958+0000","last_became_peered":"2024-05-27T00:40:30.859958+0000","last_unstale":"2024-05-27T00:40:31.846176+0000","last_undegraded":"2024-05-27T00:40:31.846176+0000","last_fullsized":"2024-05-27T00:40:31.846176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:37:28.064481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755183+0000","last_change":"2024-05-27T00:40:29.835914+0000","last_active":"2024-05-27T00:40:33.755183+0000","last_peered":"2024-05-27T00:40:33.755183+0000","last_clean":"2024-05-27T00:40:33.755183+0000","last_became_active":"2024-05-27T00:40:29.835645+0000","last_became_peered":"2024-05-27T00:40:29.835645+0000","last_unstale":"2024-05-27T00:40:33.755183+0000","last_undegraded":"2024-05-27T00:40:33.755183+0000","last_fullsized":"2024-05-27T00:40:33.755183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:25.290676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755251+0000","last_change":"2024-05-27T00:40:30.851583+0000","last_active":"2024-05-27T00:40:33.755251+0000","last_peered":"2024-05-27T00:40:33.755251+0000","last_clean":"2024-05-27T00:40:33.755251+0000","last_became_active":"2024-05-27T00:40:30.851320+0000","last_became_peered":"2024-05-27T00:40:30.851320+0000","last_unstale":"2024-05-27T00:40:33.755251+0000","last_undegraded":"2024-05-27T00:40:33.755251+0000","last_fullsized":"2024-05-27T00:40:33.755251+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:40.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846401+0000","last_change":"2024-05-27T00:40:29.839200+0000","last_active":"2024-05-27T00:40:31.846401+0000","last_peered":"2024-05-27T00:40:31.846401+0000","last_clean":"2024-05-27T00:40:31.846401+0000","last_became_active":"2024-05-27T00:40:29.838929+0000","last_became_peered":"2024-05-27T00:40:29.838929+0000","last_unstale":"2024-05-27T00:40:31.846401+0000","last_undegraded":"2024-05-27T00:40:31.846401+0000","last_fullsized":"2024-05-27T00:40:31.846401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:50.927711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755766+0000","last_change":"2024-05-27T00:40:30.843907+0000","last_active":"2024-05-27T00:40:33.755766+0000","last_peered":"2024-05-27T00:40:33.755766+0000","last_clean":"2024-05-27T00:40:33.755766+0000","last_became_active":"2024-05-27T00:40:30.842997+0000","last_became_peered":"2024-05-27T00:40:30.842997+0000","last_unstale":"2024-05-27T00:40:33.755766+0000","last_undegraded":"2024-05-27T00:40:33.755766+0000","last_fullsized":"2024-05-27T00:40:33.755766+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:11.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963897+0000","last_change":"2024-05-27T00:40:29.833098+0000","last_active":"2024-05-27T00:40:31.963897+0000","last_peered":"2024-05-27T00:40:31.963897+0000","last_clean":"2024-05-27T00:40:31.963897+0000","last_became_active":"2024-05-27T00:40:29.832815+0000","last_became_peered":"2024-05-27T00:40:29.832815+0000","last_unstale":"2024-05-27T00:40:31.963897+0000","last_undegraded":"2024-05-27T00:40:31.963897+0000","last_fullsized":"2024-05-27T00:40:31.963897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754923+0000","last_change":"2024-05-27T00:40:30.845131+0000","last_active":"2024-05-27T00:40:33.754923+0000","last_peered":"2024-05-27T00:40:33.754923+0000","last_clean":"2024-05-27T00:40:33.754923+0000","last_became_active":"2024-05-27T00:40:30.844824+0000","last_became_peered":"2024-05-27T00:40:30.844824+0000","last_unstale":"2024-05-27T00:40:33.754923+0000","last_undegraded":"2024-05-27T00:40:33.754923+0000","last_fullsized":"2024-05-27T00:40:33.754923+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964014+0000","last_change":"2024-05-27T00:40:29.836855+0000","last_active":"2024-05-27T00:40:31.964014+0000","last_peered":"2024-05-27T00:40:31.964014+0000","last_clean":"2024-05-27T00:40:31.964014+0000","last_became_active":"2024-05-27T00:40:29.836413+0000","last_became_peered":"2024-05-27T00:40:29.836413+0000","last_unstale":"2024-05-27T00:40:31.964014+0000","last_undegraded":"2024-05-27T00:40:31.964014+0000","last_fullsized":"2024-05-27T00:40:31.964014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:47.620595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756225+0000","last_change":"2024-05-27T00:40:30.847851+0000","last_active":"2024-05-27T00:40:33.756225+0000","last_peered":"2024-05-27T00:40:33.756225+0000","last_clean":"2024-05-27T00:40:33.756225+0000","last_became_active":"2024-05-27T00:40:30.847595+0000","last_became_peered":"2024-05-27T00:40:30.847595+0000","last_unstale":"2024-05-27T00:40:33.756225+0000","last_undegraded":"2024-05-27T00:40:33.756225+0000","last_fullsized":"2024-05-27T00:40:33.756225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:47:48.323260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846276+0000","last_change":"2024-05-27T00:40:29.841007+0000","last_active":"2024-05-27T00:40:31.846276+0000","last_peered":"2024-05-27T00:40:31.846276+0000","last_clean":"2024-05-27T00:40:31.846276+0000","last_became_active":"2024-05-27T00:40:29.840694+0000","last_became_peered":"2024-05-27T00:40:29.840694+0000","last_unstale":"2024-05-27T00:40:31.846276+0000","last_undegraded":"2024-05-27T00:40:31.846276+0000","last_fullsized":"2024-05-27T00:40:31.846276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:55.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754714+0000","last_change":"2024-05-27T00:40:30.853848+0000","last_active":"2024-05-27T00:40:33.754714+0000","last_peered":"2024-05-27T00:40:33.754714+0000","last_clean":"2024-05-27T00:40:33.754714+0000","last_became_active":"2024-05-27T00:40:30.853608+0000","last_became_peered":"2024-05-27T00:40:30.853608+0000","last_unstale":"2024-05-27T00:40:33.754714+0000","last_undegraded":"2024-05-27T00:40:33.754714+0000","last_fullsized":"2024-05-27T00:40:33.754714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:56.389913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846275+0000","last_change":"2024-05-27T00:40:29.832617+0000","last_active":"2024-05-27T00:40:31.846275+0000","last_peered":"2024-05-27T00:40:31.846275+0000","last_clean":"2024-05-27T00:40:31.846275+0000","last_became_active":"2024-05-27T00:40:29.832366+0000","last_became_peered":"2024-05-27T00:40:29.832366+0000","last_unstale":"2024-05-27T00:40:31.846275+0000","last_undegraded":"2024-05-27T00:40:31.846275+0000","last_fullsized":"2024-05-27T00:40:31.846275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:11.842604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846092+0000","last_change":"2024-05-27T00:40:30.855023+0000","last_active":"2024-05-27T00:40:31.846092+0000","last_peered":"2024-05-27T00:40:31.846092+0000","last_clean":"2024-05-27T00:40:31.846092+0000","last_became_active":"2024-05-27T00:40:30.854711+0000","last_became_peered":"2024-05-27T00:40:30.854711+0000","last_unstale":"2024-05-27T00:40:31.846092+0000","last_undegraded":"2024-05-27T00:40:31.846092+0000","last_fullsized":"2024-05-27T00:40:31.846092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:09:09.478467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844453+0000","last_change":"2024-05-27T00:40:29.839332+0000","last_active":"2024-05-27T00:40:31.844453+0000","last_peered":"2024-05-27T00:40:31.844453+0000","last_clean":"2024-05-27T00:40:31.844453+0000","last_became_active":"2024-05-27T00:40:29.839058+0000","last_became_peered":"2024-05-27T00:40:29.839058+0000","last_unstale":"2024-05-27T00:40:31.844453+0000","last_undegraded":"2024-05-27T00:40:31.844453+0000","last_fullsized":"2024-05-27T00:40:31.844453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755413+0000","last_change":"2024-05-27T00:40:30.842022+0000","last_active":"2024-05-27T00:40:33.755413+0000","last_peered":"2024-05-27T00:40:33.755413+0000","last_clean":"2024-05-27T00:40:33.755413+0000","last_became_active":"2024-05-27T00:40:30.841565+0000","last_became_peered":"2024-05-27T00:40:30.841565+0000","last_unstale":"2024-05-27T00:40:33.755413+0000","last_undegraded":"2024-05-27T00:40:33.755413+0000","last_fullsized":"2024-05-27T00:40:33.755413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846503+0000","last_change":"2024-05-27T00:40:29.837660+0000","last_active":"2024-05-27T00:40:31.846503+0000","last_peered":"2024-05-27T00:40:31.846503+0000","last_clean":"2024-05-27T00:40:31.846503+0000","last_became_active":"2024-05-27T00:40:29.836617+0000","last_became_peered":"2024-05-27T00:40:29.836617+0000","last_unstale":"2024-05-27T00:40:31.846503+0000","last_undegraded":"2024-05-27T00:40:31.846503+0000","last_fullsized":"2024-05-27T00:40:31.846503+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:33.427371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846221+0000","last_change":"2024-05-27T00:40:30.852888+0000","last_active":"2024-05-27T00:40:31.846221+0000","last_peered":"2024-05-27T00:40:31.846221+0000","last_clean":"2024-05-27T00:40:31.846221+0000","last_became_active":"2024-05-27T00:40:30.852614+0000","last_became_peered":"2024-05-27T00:40:30.852614+0000","last_unstale":"2024-05-27T00:40:31.846221+0000","last_undegraded":"2024-05-27T00:40:31.846221+0000","last_fullsized":"2024-05-27T00:40:31.846221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:33:24.367596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755327+0000","last_change":"2024-05-27T00:40:29.844350+0000","last_active":"2024-05-27T00:40:33.755327+0000","last_peered":"2024-05-27T00:40:33.755327+0000","last_clean":"2024-05-27T00:40:33.755327+0000","last_became_active":"2024-05-27T00:40:29.843988+0000","last_became_peered":"2024-05-27T00:40:29.843988+0000","last_unstale":"2024-05-27T00:40:33.755327+0000","last_undegraded":"2024-05-27T00:40:33.755327+0000","last_fullsized":"2024-05-27T00:40:33.755327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:37.046495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846288+0000","last_change":"2024-05-27T00:40:30.860095+0000","last_active":"2024-05-27T00:40:31.846288+0000","last_peered":"2024-05-27T00:40:31.846288+0000","last_clean":"2024-05-27T00:40:31.846288+0000","last_became_active":"2024-05-27T00:40:30.858753+0000","last_became_peered":"2024-05-27T00:40:30.858753+0000","last_unstale":"2024-05-27T00:40:31.846288+0000","last_undegraded":"2024-05-27T00:40:31.846288+0000","last_fullsized":"2024-05-27T00:40:31.846288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:12.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963918+0000","last_change":"2024-05-27T00:40:29.836227+0000","last_active":"2024-05-27T00:40:31.963918+0000","last_peered":"2024-05-27T00:40:31.963918+0000","last_clean":"2024-05-27T00:40:31.963918+0000","last_became_active":"2024-05-27T00:40:29.835837+0000","last_became_peered":"2024-05-27T00:40:29.835837+0000","last_unstale":"2024-05-27T00:40:31.963918+0000","last_undegraded":"2024-05-27T00:40:31.963918+0000","last_fullsized":"2024-05-27T00:40:31.963918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:25:25.290676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963969+0000","last_change":"2024-05-27T00:40:30.848394+0000","last_active":"2024-05-27T00:40:31.963969+0000","last_peered":"2024-05-27T00:40:31.963969+0000","last_clean":"2024-05-27T00:40:31.963969+0000","last_became_active":"2024-05-27T00:40:30.846482+0000","last_became_peered":"2024-05-27T00:40:30.846482+0000","last_unstale":"2024-05-27T00:40:31.963969+0000","last_undegraded":"2024-05-27T00:40:31.963969+0000","last_fullsized":"2024-05-27T00:40:31.963969+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:12.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756289+0000","last_change":"2024-05-27T00:40:29.838407+0000","last_active":"2024-05-27T00:40:33.756289+0000","last_peered":"2024-05-27T00:40:33.756289+0000","last_clean":"2024-05-27T00:40:33.756289+0000","last_became_active":"2024-05-27T00:40:29.837279+0000","last_became_peered":"2024-05-27T00:40:29.837279+0000","last_unstale":"2024-05-27T00:40:33.756289+0000","last_undegraded":"2024-05-27T00:40:33.756289+0000","last_fullsized":"2024-05-27T00:40:33.756289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:05.797447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756221+0000","last_change":"2024-05-27T00:40:30.846070+0000","last_active":"2024-05-27T00:40:33.756221+0000","last_peered":"2024-05-27T00:40:33.756221+0000","last_clean":"2024-05-27T00:40:33.756221+0000","last_became_active":"2024-05-27T00:40:30.845813+0000","last_became_peered":"2024-05-27T00:40:30.845813+0000","last_unstale":"2024-05-27T00:40:33.756221+0000","last_undegraded":"2024-05-27T00:40:33.756221+0000","last_fullsized":"2024-05-27T00:40:33.756221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755160+0000","last_change":"2024-05-27T00:40:29.836358+0000","last_active":"2024-05-27T00:40:33.755160+0000","last_peered":"2024-05-27T00:40:33.755160+0000","last_clean":"2024-05-27T00:40:33.755160+0000","last_became_active":"2024-05-27T00:40:29.836030+0000","last_became_peered":"2024-05-27T00:40:29.836030+0000","last_unstale":"2024-05-27T00:40:33.755160+0000","last_undegraded":"2024-05-27T00:40:33.755160+0000","last_fullsized":"2024-05-27T00:40:33.755160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:31.410996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754925+0000","last_change":"2024-05-27T00:40:30.858900+0000","last_active":"2024-05-27T00:40:33.754925+0000","last_peered":"2024-05-27T00:40:33.754925+0000","last_clean":"2024-05-27T00:40:33.754925+0000","last_became_active":"2024-05-27T00:40:30.858588+0000","last_became_peered":"2024-05-27T00:40:30.858588+0000","last_unstale":"2024-05-27T00:40:33.754925+0000","last_undegraded":"2024-05-27T00:40:33.754925+0000","last_fullsized":"2024-05-27T00:40:33.754925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:36.744482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844468+0000","last_change":"2024-05-27T00:40:29.843381+0000","last_active":"2024-05-27T00:40:31.844468+0000","last_peered":"2024-05-27T00:40:31.844468+0000","last_clean":"2024-05-27T00:40:31.844468+0000","last_became_active":"2024-05-27T00:40:29.843087+0000","last_became_peered":"2024-05-27T00:40:29.843087+0000","last_unstale":"2024-05-27T00:40:31.844468+0000","last_undegraded":"2024-05-27T00:40:31.844468+0000","last_fullsized":"2024-05-27T00:40:31.844468+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:48.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755991+0000","last_change":"2024-05-27T00:40:30.844974+0000","last_active":"2024-05-27T00:40:33.755991+0000","last_peered":"2024-05-27T00:40:33.755991+0000","last_clean":"2024-05-27T00:40:33.755991+0000","last_became_active":"2024-05-27T00:40:30.843119+0000","last_became_peered":"2024-05-27T00:40:30.843119+0000","last_unstale":"2024-05-27T00:40:33.755991+0000","last_undegraded":"2024-05-27T00:40:33.755991+0000","last_fullsized":"2024-05-27T00:40:33.755991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:20:33.841686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754605+0000","last_change":"2024-05-27T00:40:29.831709+0000","last_active":"2024-05-27T00:40:33.754605+0000","last_peered":"2024-05-27T00:40:33.754605+0000","last_clean":"2024-05-27T00:40:33.754605+0000","last_became_active":"2024-05-27T00:40:29.831355+0000","last_became_peered":"2024-05-27T00:40:29.831355+0000","last_unstale":"2024-05-27T00:40:33.754605+0000","last_undegraded":"2024-05-27T00:40:33.754605+0000","last_fullsized":"2024-05-27T00:40:33.754605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:12.195311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754862+0000","last_change":"2024-05-27T00:40:30.844565+0000","last_active":"2024-05-27T00:40:33.754862+0000","last_peered":"2024-05-27T00:40:33.754862+0000","last_clean":"2024-05-27T00:40:33.754862+0000","last_became_active":"2024-05-27T00:40:30.844279+0000","last_became_peered":"2024-05-27T00:40:30.844279+0000","last_unstale":"2024-05-27T00:40:33.754862+0000","last_undegraded":"2024-05-27T00:40:33.754862+0000","last_fullsized":"2024-05-27T00:40:33.754862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:12.844381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754669+0000","last_change":"2024-05-27T00:40:29.835427+0000","last_active":"2024-05-27T00:40:33.754669+0000","last_peered":"2024-05-27T00:40:33.754669+0000","last_clean":"2024-05-27T00:40:33.754669+0000","last_became_active":"2024-05-27T00:40:29.834727+0000","last_became_peered":"2024-05-27T00:40:29.834727+0000","last_unstale":"2024-05-27T00:40:33.754669+0000","last_undegraded":"2024-05-27T00:40:33.754669+0000","last_fullsized":"2024-05-27T00:40:33.754669+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:58:42.956086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963716+0000","last_change":"2024-05-27T00:40:30.851594+0000","last_active":"2024-05-27T00:40:31.963716+0000","last_peered":"2024-05-27T00:40:31.963716+0000","last_clean":"2024-05-27T00:40:31.963716+0000","last_became_active":"2024-05-27T00:40:30.851318+0000","last_became_peered":"2024-05-27T00:40:30.851318+0000","last_unstale":"2024-05-27T00:40:31.963716+0000","last_undegraded":"2024-05-27T00:40:31.963716+0000","last_fullsized":"2024-05-27T00:40:31.963716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:23:13.579222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755911+0000","last_change":"2024-05-27T00:40:29.843742+0000","last_active":"2024-05-27T00:40:33.755911+0000","last_peered":"2024-05-27T00:40:33.755911+0000","last_clean":"2024-05-27T00:40:33.755911+0000","last_became_active":"2024-05-27T00:40:29.843445+0000","last_became_peered":"2024-05-27T00:40:29.843445+0000","last_unstale":"2024-05-27T00:40:33.755911+0000","last_undegraded":"2024-05-27T00:40:33.755911+0000","last_fullsized":"2024-05-27T00:40:33.755911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846350+0000","last_change":"2024-05-27T00:40:30.854867+0000","last_active":"2024-05-27T00:40:31.846350+0000","last_peered":"2024-05-27T00:40:31.846350+0000","last_clean":"2024-05-27T00:40:31.846350+0000","last_became_active":"2024-05-27T00:40:30.854579+0000","last_became_peered":"2024-05-27T00:40:30.854579+0000","last_unstale":"2024-05-27T00:40:31.846350+0000","last_undegraded":"2024-05-27T00:40:31.846350+0000","last_fullsized":"2024-05-27T00:40:31.846350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:06:27.438392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754435+0000","last_change":"2024-05-27T00:40:29.830333+0000","last_active":"2024-05-27T00:40:33.754435+0000","last_peered":"2024-05-27T00:40:33.754435+0000","last_clean":"2024-05-27T00:40:33.754435+0000","last_became_active":"2024-05-27T00:40:29.830050+0000","last_became_peered":"2024-05-27T00:40:29.830050+0000","last_unstale":"2024-05-27T00:40:33.754435+0000","last_undegraded":"2024-05-27T00:40:33.754435+0000","last_fullsized":"2024-05-27T00:40:33.754435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:01.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844646+0000","last_change":"2024-05-27T00:40:30.851081+0000","last_active":"2024-05-27T00:40:31.844646+0000","last_peered":"2024-05-27T00:40:31.844646+0000","last_clean":"2024-05-27T00:40:31.844646+0000","last_became_active":"2024-05-27T00:40:30.850755+0000","last_became_peered":"2024-05-27T00:40:30.850755+0000","last_unstale":"2024-05-27T00:40:31.844646+0000","last_undegraded":"2024-05-27T00:40:31.844646+0000","last_fullsized":"2024-05-27T00:40:31.844646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:11.868864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.964070+0000","last_change":"2024-05-27T00:40:29.835772+0000","last_active":"2024-05-27T00:40:31.964070+0000","last_peered":"2024-05-27T00:40:31.964070+0000","last_clean":"2024-05-27T00:40:31.964070+0000","last_became_active":"2024-05-27T00:40:29.835453+0000","last_became_peered":"2024-05-27T00:40:29.835453+0000","last_unstale":"2024-05-27T00:40:31.964070+0000","last_undegraded":"2024-05-27T00:40:31.964070+0000","last_fullsized":"2024-05-27T00:40:31.964070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:05.797447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756274+0000","last_change":"2024-05-27T00:40:30.844109+0000","last_active":"2024-05-27T00:40:33.756274+0000","last_peered":"2024-05-27T00:40:33.756274+0000","last_clean":"2024-05-27T00:40:33.756274+0000","last_became_active":"2024-05-27T00:40:30.842384+0000","last_became_peered":"2024-05-27T00:40:30.842384+0000","last_unstale":"2024-05-27T00:40:33.756274+0000","last_undegraded":"2024-05-27T00:40:33.756274+0000","last_fullsized":"2024-05-27T00:40:33.756274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:51.877600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755046+0000","last_change":"2024-05-27T00:40:29.827858+0000","last_active":"2024-05-27T00:40:33.755046+0000","last_peered":"2024-05-27T00:40:33.755046+0000","last_clean":"2024-05-27T00:40:33.755046+0000","last_became_active":"2024-05-27T00:40:29.827589+0000","last_became_peered":"2024-05-27T00:40:29.827589+0000","last_unstale":"2024-05-27T00:40:33.755046+0000","last_undegraded":"2024-05-27T00:40:33.755046+0000","last_fullsized":"2024-05-27T00:40:33.755046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:28.730539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846318+0000","last_change":"2024-05-27T00:40:30.860768+0000","last_active":"2024-05-27T00:40:31.846318+0000","last_peered":"2024-05-27T00:40:31.846318+0000","last_clean":"2024-05-27T00:40:31.846318+0000","last_became_active":"2024-05-27T00:40:30.860451+0000","last_became_peered":"2024-05-27T00:40:30.860451+0000","last_unstale":"2024-05-27T00:40:31.846318+0000","last_undegraded":"2024-05-27T00:40:31.846318+0000","last_fullsized":"2024-05-27T00:40:31.846318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:43:23.638293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755169+0000","last_change":"2024-05-27T00:40:29.835183+0000","last_active":"2024-05-27T00:40:33.755169+0000","last_peered":"2024-05-27T00:40:33.755169+0000","last_clean":"2024-05-27T00:40:33.755169+0000","last_became_active":"2024-05-27T00:40:29.834922+0000","last_became_peered":"2024-05-27T00:40:29.834922+0000","last_unstale":"2024-05-27T00:40:33.755169+0000","last_undegraded":"2024-05-27T00:40:33.755169+0000","last_fullsized":"2024-05-27T00:40:33.755169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:28.086108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754715+0000","last_change":"2024-05-27T00:40:30.859747+0000","last_active":"2024-05-27T00:40:33.754715+0000","last_peered":"2024-05-27T00:40:33.754715+0000","last_clean":"2024-05-27T00:40:33.754715+0000","last_became_active":"2024-05-27T00:40:30.859488+0000","last_became_peered":"2024-05-27T00:40:30.859488+0000","last_unstale":"2024-05-27T00:40:33.754715+0000","last_undegraded":"2024-05-27T00:40:33.754715+0000","last_fullsized":"2024-05-27T00:40:33.754715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:36:51.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.754941+0000","last_change":"2024-05-27T00:40:29.837690+0000","last_active":"2024-05-27T00:40:33.754941+0000","last_peered":"2024-05-27T00:40:33.754941+0000","last_clean":"2024-05-27T00:40:33.754941+0000","last_became_active":"2024-05-27T00:40:29.837411+0000","last_became_peered":"2024-05-27T00:40:29.837411+0000","last_unstale":"2024-05-27T00:40:33.754941+0000","last_undegraded":"2024-05-27T00:40:33.754941+0000","last_fullsized":"2024-05-27T00:40:33.754941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:42:39.848002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.844355+0000","last_change":"2024-05-27T00:40:30.849204+0000","last_active":"2024-05-27T00:40:31.844355+0000","last_peered":"2024-05-27T00:40:31.844355+0000","last_clean":"2024-05-27T00:40:31.844355+0000","last_became_active":"2024-05-27T00:40:30.848866+0000","last_became_peered":"2024-05-27T00:40:30.848866+0000","last_unstale":"2024-05-27T00:40:31.844355+0000","last_undegraded":"2024-05-27T00:40:31.844355+0000","last_fullsized":"2024-05-27T00:40:31.844355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:56.389913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846650+0000","last_change":"2024-05-27T00:40:29.833805+0000","last_active":"2024-05-27T00:40:31.846650+0000","last_peered":"2024-05-27T00:40:31.846650+0000","last_clean":"2024-05-27T00:40:31.846650+0000","last_became_active":"2024-05-27T00:40:29.833515+0000","last_became_peered":"2024-05-27T00:40:29.833515+0000","last_unstale":"2024-05-27T00:40:31.846650+0000","last_undegraded":"2024-05-27T00:40:31.846650+0000","last_fullsized":"2024-05-27T00:40:31.846650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:05.002271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755459+0000","last_change":"2024-05-27T00:40:30.854612+0000","last_active":"2024-05-27T00:40:33.755459+0000","last_peered":"2024-05-27T00:40:33.755459+0000","last_clean":"2024-05-27T00:40:33.755459+0000","last_became_active":"2024-05-27T00:40:30.854334+0000","last_became_peered":"2024-05-27T00:40:30.854334+0000","last_unstale":"2024-05-27T00:40:33.755459+0000","last_undegraded":"2024-05-27T00:40:33.755459+0000","last_fullsized":"2024-05-27T00:40:33.755459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:45.727676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755280+0000","last_change":"2024-05-27T00:40:29.834692+0000","last_active":"2024-05-27T00:40:33.755280+0000","last_peered":"2024-05-27T00:40:33.755280+0000","last_clean":"2024-05-27T00:40:33.755280+0000","last_became_active":"2024-05-27T00:40:29.834412+0000","last_became_peered":"2024-05-27T00:40:29.834412+0000","last_unstale":"2024-05-27T00:40:33.755280+0000","last_undegraded":"2024-05-27T00:40:33.755280+0000","last_fullsized":"2024-05-27T00:40:33.755280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:02.837963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756089+0000","last_change":"2024-05-27T00:40:30.845502+0000","last_active":"2024-05-27T00:40:33.756089+0000","last_peered":"2024-05-27T00:40:33.756089+0000","last_clean":"2024-05-27T00:40:33.756089+0000","last_became_active":"2024-05-27T00:40:30.843028+0000","last_became_peered":"2024-05-27T00:40:30.843028+0000","last_unstale":"2024-05-27T00:40:33.756089+0000","last_undegraded":"2024-05-27T00:40:33.756089+0000","last_fullsized":"2024-05-27T00:40:33.756089+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:03:43.297585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.846039+0000","last_change":"2024-05-27T00:40:29.835197+0000","last_active":"2024-05-27T00:40:31.846039+0000","last_peered":"2024-05-27T00:40:31.846039+0000","last_clean":"2024-05-27T00:40:31.846039+0000","last_became_active":"2024-05-27T00:40:29.834907+0000","last_became_peered":"2024-05-27T00:40:29.834907+0000","last_unstale":"2024-05-27T00:40:31.846039+0000","last_undegraded":"2024-05-27T00:40:31.846039+0000","last_fullsized":"2024-05-27T00:40:31.846039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:15:36.781562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:31.963924+0000","last_change":"2024-05-27T00:40:30.859603+0000","last_active":"2024-05-27T00:40:31.963924+0000","last_peered":"2024-05-27T00:40:31.963924+0000","last_clean":"2024-05-27T00:40:31.963924+0000","last_became_active":"2024-05-27T00:40:30.859333+0000","last_became_peered":"2024-05-27T00:40:30.859333+0000","last_unstale":"2024-05-27T00:40:31.963924+0000","last_undegraded":"2024-05-27T00:40:31.963924+0000","last_fullsized":"2024-05-27T00:40:31.963924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:29.817168+0000","last_clean_scrub_stamp":"2024-05-27T00:40:29.817168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:16.405801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.756215+0000","last_change":"2024-05-27T00:40:29.843470+0000","last_active":"2024-05-27T00:40:33.756215+0000","last_peered":"2024-05-27T00:40:33.756215+0000","last_clean":"2024-05-27T00:40:33.756215+0000","last_became_active":"2024-05-27T00:40:29.843159+0000","last_became_peered":"2024-05-27T00:40:29.843159+0000","last_unstale":"2024-05-27T00:40:33.756215+0000","last_undegraded":"2024-05-27T00:40:33.756215+0000","last_fullsized":"2024-05-27T00:40:33.756215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:40:28.815391+0000","last_clean_scrub_stamp":"2024-05-27T00:40:28.815391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:15:48.334664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T00:40:33.755746+0000","last_change":"2024-05-27T00:39:10.216657+0000","last_active":"2024-05-27T00:40:33.755746+0000","last_peered":"2024-05-27T00:40:33.755746+0000","last_clean":"2024-05-27T00:40:33.755746+0000","last_became_active":"2024-05-27T00:39:10.215631+0000","last_became_peered":"2024-05-27T00:39:10.215631+0000","last_unstale":"2024-05-27T00:40:33.755746+0000","last_undegraded":"2024-05-27T00:40:33.755746+0000","last_fullsized":"2024-05-27T00:40:33.755746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T00:39:08.976609+0000","last_clean_scrub_stamp":"2024-05-27T00:39:08.976609+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:17:10.766592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":197909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":197439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":198973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":197417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":781037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T00:40:39.074 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-27T00:40:39.075 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-27T00:40:39.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 Processor -- start 2024-05-27T00:40:39.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 -- start start 2024-05-27T00:40:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 0x7fdd041459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fdd04070030 0x7fdd040704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd04145f90 con 0x7fdd0415e830 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fdd04146270 con 0x7fdd04070030 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.216+0000 7fdd09529640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd04146550 con 0x7fdd04144a10 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd027fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdd0415e830 0x7fdd040a9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:59002/0 (socket says 172.21.15.111:59002) 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd027fc640 1 -- 172.21.15.111:0/697590399 learned_addr learned my addr 172.21.15.111:0/697590399 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874107249 0 0) 0x7fdd04145f90 con 0x7fdd0415e830 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcec003520 con 0x7fdd0415e830 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd02ffd640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 0x7fdd041459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd037fe640 1 --2- 172.21.15.111:0/697590399 >> v2:172.21.15.153:3300/0 conn(0x7fdd04070030 0x7fdd040704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2891524898 0 0) 0x7fdcec003520 con 0x7fdd0415e830 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 msgr2=0x7fdd041459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 0x7fdd041459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 >> v2:172.21.15.153:3300/0 conn(0x7fdd04070030 msgr2=0x7fdd040704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:39.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 --2- 172.21.15.111:0/697590399 >> v2:172.21.15.153:3300/0 conn(0x7fdd04070030 0x7fdd040704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd04146830 con 0x7fdd0415e830 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fdcf8001670 con 0x7fdd0415e830 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 >> v1:172.21.15.111:6789/0 conn(0x7fdd0415e830 legacy=0x7fdd040a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 0x7fdcec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec003fe0 0x7fdcec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcec004d40 0x7fdcec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04146270 con 0x7fdcec004d40 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04146550 con 0x7fdcec003fe0 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec00bac0 con 0x7fdd04144a10 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd037fe640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcec004d40 0x7fdcec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd02ffd640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 0x7fdcec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.217+0000 7fdd027fc640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec003fe0 0x7fdcec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd037fe640 1 -- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 msgr2=0x7fdcec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd037fe640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd04144a10 0x7fdcec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd037fe640 1 -- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec003fe0 msgr2=0x7fdcec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd037fe640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec003fe0 0x7fdcec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd037fe640 1 -- 172.21.15.111:0/697590399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec003520 con 0x7fdcec004d40 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd037fe640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcec004d40 0x7fdcec0051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc007360 tx=0x7fdcec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b80196440342a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc009be0 con 0x7fdcec004d40 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcfc02e620 con 0x7fdcec004d40 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.218+0000 7fdd01ffb640 1 -- 172.21.15.111:0/697590399 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc02e9f0 con 0x7fdcec004d40 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd09529640 1 -- 172.21.15.111:0/697590399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcec004d40 msgr2=0x7fdcec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd09529640 1 --2- 172.21.15.111:0/697590399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcec004d40 0x7fdcec0051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc007360 tx=0x7fdcec005830 comp rx=0 tx=0).stop 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd037fe640 1 -- 172.21.15.111:0/697590399 reap_dead start 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd09529640 1 -- 172.21.15.111:0/697590399 shutdown_connections 2024-05-27T00:40:39.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd09529640 1 -- 172.21.15.111:0/697590399 >> 172.21.15.111:0/697590399 conn(0x7fdd04073130 msgr2=0x7fdd040b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:39.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd09529640 1 -- 172.21.15.111:0/697590399 shutdown_connections 2024-05-27T00:40:39.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.219+0000 7fdd09529640 1 -- 172.21.15.111:0/697590399 wait complete. 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.220+0000 7fdd09529640 1 Processor -- start 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.220+0000 7fdd09529640 1 -- start start 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd09529640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec004d40 0x7fdd041f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd09529640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 0x7fdd040cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd09529640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd040b0b80 0x7fdd040b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd09529640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04145b30 con 0x7fdd041f1e50 2024-05-27T00:40:39.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd09529640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fdd041459b0 con 0x7fdcec004d40 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd09529640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fdd04145cb0 con 0x7fdd040b0b80 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd027fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 0x7fdd040cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd027fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 0x7fdd040cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59570/0 (socket says 172.21.15.111:59570) 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd027fc640 1 -- 172.21.15.111:0/3272506167 learned_addr learned my addr 172.21.15.111:0/3272506167 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd02ffd640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec004d40 0x7fdd041f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd037fe640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd040b0b80 0x7fdd040b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd027fc640 1 -- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd040b0b80 msgr2=0x7fdd040b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd027fc640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd040b0b80 0x7fdd040b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.221+0000 7fdd027fc640 1 -- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec004d40 msgr2=0x7fdd041f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdd027fc640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec004d40 0x7fdd041f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdd027fc640 1 -- 172.21.15.111:0/3272506167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd041f51d0 con 0x7fdd041f1e50 2024-05-27T00:40:39.227 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdd027fc640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 0x7fdd040cf6f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007320 tx=0x7fdcf80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf801a070 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf800f9a0 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd041f5700 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800fd70 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.222+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd041f5cc0 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.223+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd04068680 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.223+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcf801d070 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.224+0000 7fdceb7fe640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdcf006be60 0x7fdcf006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.224+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdcf808aa10 con 0x7fdd041f1e50 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.224+0000 7fdd02ffd640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdcf006be60 0x7fdcf006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.225+0000 7fdd02ffd640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdcf006be60 0x7fdcf006e320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4006fd0 tx=0x7fdcf4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:39.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.228+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdd041f51d0 con 0x7fdd041f1e50 2024-05-27T00:40:39.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.465+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdd040b5550 con 0x7fdd041f1e50 2024-05-27T00:40:39.468 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.467+0000 7fdceb7fe640 1 -- 172.21.15.111:0/3272506167 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7fdcf8059230 con 0x7fdd041f1e50 2024-05-27T00:40:39.468 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-05-27T00:40:39.468 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD61VNmQ9y6FBAAIeW3Ce0vjSxE/AG+cOZaUQ== 2024-05-27T00:40:39.469 INFO:teuthology.orchestra.run.smithi111.stdout: caps mds = "allow" 2024-05-27T00:40:39.469 INFO:teuthology.orchestra.run.smithi111.stdout: caps mon = "allow r" 2024-05-27T00:40:39.469 INFO:teuthology.orchestra.run.smithi111.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-27T00:40:39.469 INFO:teuthology.orchestra.run.smithi111.stderr:updated caps for client.0 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdcf006be60 msgr2=0x7fdcf006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdcf006be60 0x7fdcf006e320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4006fd0 tx=0x7fdcf4008040 comp rx=0 tx=0).stop 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 msgr2=0x7fdd040cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 0x7fdd040cf6f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007320 tx=0x7fdcf80077e0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 shutdown_connections 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fdcf006be60 0x7fdcf006e320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fdd040b0b80 0x7fdd040b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd041f1e50 0x7fdd040cf6f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 --2- 172.21.15.111:0/3272506167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcec004d40 0x7fdd041f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 >> 172.21.15.111:0/3272506167 conn(0x7fdd04073130 msgr2=0x7fdd04140760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 shutdown_connections 2024-05-27T00:40:39.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.470+0000 7fdd09529640 1 -- 172.21.15.111:0/3272506167 wait complete. 2024-05-27T00:40:39.482 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T00:40:39.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.580+0000 7f2d3bd34640 1 Processor -- start 2024-05-27T00:40:39.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.580+0000 7f2d3bd34640 1 -- start start 2024-05-27T00:40:39.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3bd34640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2d34144000 0x7f2d340b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3bd34640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2d3406b390 0x7f2d340cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3bd34640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d340cdc80 0x7f2d340726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3bd34640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7f2d340b42f0 con 0x7f2d34144000 2024-05-27T00:40:39.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3bd34640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f2d340b3440 con 0x7f2d3406b390 2024-05-27T00:40:39.584 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3bd34640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2d340b36e0 con 0x7f2d340cdc80 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2d34144000 0x7f2d340b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2d34144000 0x7f2d340b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59572/0 (socket says 172.21.15.111:59572) 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 -- 172.21.15.111:0/396818418 learned_addr learned my addr 172.21.15.111:0/396818418 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d392cf640 1 --2- 172.21.15.111:0/396818418 >> v2:172.21.15.153:3300/0 conn(0x7f2d3406b390 0x7f2d340cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d3a2d1640 1 --2- 172.21.15.111:0/396818418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d340cdc80 0x7f2d340726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 -- 172.21.15.111:0/396818418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d340cdc80 msgr2=0x7f2d340726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 --2- 172.21.15.111:0/396818418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d340cdc80 0x7f2d340726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 -- 172.21.15.111:0/396818418 >> v2:172.21.15.153:3300/0 conn(0x7f2d3406b390 msgr2=0x7f2d340cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 --2- 172.21.15.111:0/396818418 >> v2:172.21.15.153:3300/0 conn(0x7f2d3406b390 0x7f2d340cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.581+0000 7f2d39ad0640 1 -- 172.21.15.111:0/396818418 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d340b39c0 con 0x7f2d34144000 2024-05-27T00:40:39.585 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.582+0000 7f2d39ad0640 1 --2- 172.21.15.111:0/396818418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d34144000 0x7f2d340b2f00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2d34072650 tx=0x7f2d3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a81ee52c69db0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.582+0000 7f2d38ace640 1 -- 172.21.15.111:0/396818418 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30037070 con 0x7f2d34144000 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.582+0000 7f2d38ace640 1 -- 172.21.15.111:0/396818418 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30032440 con 0x7f2d34144000 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.582+0000 7f2d38ace640 1 -- 172.21.15.111:0/396818418 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30032810 con 0x7f2d34144000 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 -- 172.21.15.111:0/396818418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d34144000 msgr2=0x7f2d340b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/396818418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d34144000 0x7f2d340b2f00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2d34072650 tx=0x7f2d3002a1e0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 -- 172.21.15.111:0/396818418 shutdown_connections 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/396818418 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d340cdc80 0x7f2d340726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/396818418 >> v2:172.21.15.153:3300/0 conn(0x7f2d3406b390 0x7f2d340cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/396818418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d34144000 0x7f2d340b2f00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.586 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 -- 172.21.15.111:0/396818418 >> 172.21.15.111:0/396818418 conn(0x7f2d340b1c00 msgr2=0x7f2d340b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:39.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 -- 172.21.15.111:0/396818418 shutdown_connections 2024-05-27T00:40:39.587 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.583+0000 7f2d3bd34640 1 -- 172.21.15.111:0/396818418 wait complete. 2024-05-27T00:40:39.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.584+0000 7f2d3bd34640 1 Processor -- start 2024-05-27T00:40:39.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.584+0000 7f2d3bd34640 1 -- start start 2024-05-27T00:40:39.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d3406b390 0x7f2d340c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 0x7f2d34159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d34144000 0x7f2d340ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34072c10 con 0x7f2d340cdc80 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f2d340b3f80 con 0x7f2d34144000 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f2d34072d90 con 0x7f2d3406b390 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 0x7f2d34159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 0x7f2d34159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59588/0 (socket says 172.21.15.111:59588) 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 -- 172.21.15.111:0/1638410161 learned_addr learned my addr 172.21.15.111:0/1638410161 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3a2d1640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d34144000 0x7f2d340ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d39ad0640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d3406b390 0x7f2d340c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 -- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d3406b390 msgr2=0x7f2d340c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d3406b390 0x7f2d340c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 -- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d34144000 msgr2=0x7f2d340ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d34144000 0x7f2d340ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 -- 172.21.15.111:0/1638410161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d340c8940 con 0x7f2d340cdc80 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3a2d1640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d34144000 0x7f2d340ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d392cf640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 0x7f2d34159ec0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400d800 tx=0x7f2d2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24018030 con 0x7f2d340cdc80 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24004020 con 0x7f2d340cdc80 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d340c8c20 con 0x7f2d340cdc80 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.585+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240041f0 con 0x7f2d340cdc80 2024-05-27T00:40:39.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.586+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d340c91e0 con 0x7f2d340cdc80 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.587+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d34068680 con 0x7f2d340cdc80 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.587+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d2401b070 con 0x7f2d340cdc80 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.587+0000 7f2d2a7fc640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2d1806be30 0x7f2d1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.588+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2d2408b130 con 0x7f2d340cdc80 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.588+0000 7f2d39ad0640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2d1806be30 0x7f2d1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.588+0000 7f2d39ad0640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2d1806be30 0x7f2d1806e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d30034000 tx=0x7f2d3002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:39.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.592+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2d240599e0 con 0x7f2d340cdc80 2024-05-27T00:40:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.799+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d340cd700 con 0x7f2d340cdc80 2024-05-27T00:40:39.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.799+0000 7f2d2a7fc640 1 -- 172.21.15.111:0/1638410161 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5190 (secure 0 0 0) 0x7f2d240592e0 con 0x7f2d340cdc80 2024-05-27T00:40:39.800 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:39.801 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1589050951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1589050951},{"type":"v1","addr":"172.21.15.153:6836","nonce":1589050951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5035,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/662515803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":662515803},{"type":"v1","addr":"172.21.15.153:6837","nonce":662515803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6834/1197288581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1197288581},{"type":"v1","addr":"172.21.15.111:6834","nonce":1197288581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T00:40:30.176062+0000","modified":"2024-05-27T00:40:32.738961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6835/3118800442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6833","nonce":3118800442},{"type":"v1","addr":"172.21.15.111:6835","nonce":3118800442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5418,"qdb_cluster":[5418]},"id":2}]} 2024-05-27T00:40:39.801 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 17 2024-05-27T00:40:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2d1806be30 msgr2=0x7f2d1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2d1806be30 0x7f2d1806e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d30034000 tx=0x7f2d3002e000 comp rx=0 tx=0).stop 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 msgr2=0x7f2d34159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 0x7f2d34159ec0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400d800 tx=0x7f2d2400dcc0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 shutdown_connections 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f2d1806be30 0x7f2d1806e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d34144000 0x7f2d340ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d340cdc80 0x7f2d34159ec0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 --2- 172.21.15.111:0/1638410161 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f2d3406b390 0x7f2d340c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 >> 172.21.15.111:0/1638410161 conn(0x7f2d340b1c00 msgr2=0x7f2d340b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 shutdown_connections 2024-05-27T00:40:39.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:39.801+0000 7f2d3bd34640 1 -- 172.21.15.111:0/1638410161 wait complete. 2024-05-27T00:40:40.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T00:40:30.176062+0000', 'modified': '2024-05-27T00:40:32.738961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5418': {'gid': 5418, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6835/3118800442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6833', 'nonce': 3118800442}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 3118800442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5418, 'qdb_cluster': [5418]} 2024-05-27T00:40:40.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T00:40:40.216 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-27T00:40:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.319+0000 7f0d369b0640 1 Processor -- start 2024-05-27T00:40:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.319+0000 7f0d369b0640 1 -- start start 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d369b0640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 0x7f0d300cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d369b0640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 0x7f0d300cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d369b0640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d30142650 con 0x7f0d300cce00 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d369b0640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f0d300cf210 con 0x7f0d3013f340 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d369b0640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0d300cebc0 con 0x7f0d3013ef60 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d27fff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 0x7f0d300cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 0x7f0d300cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d27fff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 0x7f0d300cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:50262/0 (socket says 172.21.15.111:50262) 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 0x7f0d300cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:52990/0 (socket says 172.21.15.111:52990) 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d27fff640 1 -- 172.21.15.111:0/1609015456 learned_addr learned my addr 172.21.15.111:0/1609015456 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d34f4d640 1 --1- 172.21.15.111:0/1609015456 >> v1:172.21.15.153:6789/0 conn(0x7f0d300cce00 0x7f0d30141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:41666/0 (socket says 172.21.15.111:41666) 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d2ffff640 1 -- 172.21.15.111:0/1609015456 >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 msgr2=0x7f0d300cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/1609015456 >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 0x7f0d300cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d2ffff640 1 -- 172.21.15.111:0/1609015456 >> v1:172.21.15.153:6789/0 conn(0x7f0d300cce00 legacy=0x7f0d30141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d2ffff640 1 -- 172.21.15.111:0/1609015456 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d300ceea0 con 0x7f0d3013ef60 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.320+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1609015456 >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 0x7f0d300cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.321+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/1609015456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 0x7f0d300cf6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d200097a0 tx=0x7f0d2002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b330128f657b4ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.321+0000 7f0d2f7fe640 1 -- 172.21.15.111:0/1609015456 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20037070 con 0x7f0d3013ef60 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.321+0000 7f0d2f7fe640 1 -- 172.21.15.111:0/1609015456 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d20032440 con 0x7f0d3013ef60 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.321+0000 7f0d2f7fe640 1 -- 172.21.15.111:0/1609015456 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20032810 con 0x7f0d3013ef60 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 -- 172.21.15.111:0/1609015456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 msgr2=0x7f0d300cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 --2- 172.21.15.111:0/1609015456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 0x7f0d300cf6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d200097a0 tx=0x7f0d2002a180 comp rx=0 tx=0).stop 2024-05-27T00:40:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 -- 172.21.15.111:0/1609015456 shutdown_connections 2024-05-27T00:40:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 --2- 172.21.15.111:0/1609015456 >> v2:172.21.15.153:3300/0 conn(0x7f0d3013f340 0x7f0d300cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 --2- 172.21.15.111:0/1609015456 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d3013ef60 0x7f0d300cf6a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 -- 172.21.15.111:0/1609015456 >> 172.21.15.111:0/1609015456 conn(0x7f0d300b1c00 msgr2=0x7f0d300b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 -- 172.21.15.111:0/1609015456 shutdown_connections 2024-05-27T00:40:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.322+0000 7f0d369b0640 1 -- 172.21.15.111:0/1609015456 wait complete. 2024-05-27T00:40:40.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.323+0000 7f0d369b0640 1 Processor -- start 2024-05-27T00:40:40.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.323+0000 7f0d369b0640 1 -- start start 2024-05-27T00:40:40.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d369b0640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d300cce00 0x7f0d3013e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d369b0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 0x7f0d300b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d369b0640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d3013f340 0x7f0d3006b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d369b0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0d30142030 con 0x7f0d3013ef60 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d369b0640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d300ce890 con 0x7f0d3013f340 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d369b0640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0d301421b0 con 0x7f0d300cce00 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 0x7f0d300b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 0x7f0d300b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59596/0 (socket says 172.21.15.111:59596) 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 -- 172.21.15.111:0/2162326010 learned_addr learned my addr 172.21.15.111:0/2162326010 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d3013f340 0x7f0d3006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 -- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d300cce00 msgr2=0x7f0d3013e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d300cce00 0x7f0d3013e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 -- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d3013f340 msgr2=0x7f0d3006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d3013f340 0x7f0d3006b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 -- 172.21.15.111:0/2162326010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3013ed80 con 0x7f0d3013ef60 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.324+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 0x7f0d300b3000 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800e9b0 tx=0x7f0d1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.328 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.325+0000 7f0d2d7fa640 1 -- 172.21.15.111:0/2162326010 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18017070 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.325+0000 7f0d2d7fa640 1 -- 172.21.15.111:0/2162326010 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1800c9a0 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.325+0000 7f0d2d7fa640 1 -- 172.21.15.111:0/2162326010 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800cd50 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.325+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d300b3850 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.325+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d300b3e10 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.326+0000 7f0d2d7fa640 1 -- 172.21.15.111:0/2162326010 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d1801a070 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.326+0000 7f0d2d7fa640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0d1006be30 0x7f0d1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.327+0000 7f0d2d7fa640 1 -- 172.21.15.111:0/2162326010 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0d1808aea0 con 0x7f0d3013ef60 2024-05-27T00:40:40.329 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.327+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0d1006be30 0x7f0d1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.327+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0d1006be30 0x7f0d1006e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0d20034000 tx=0x7f0d2002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.327+0000 7f0d3979e740 1 Processor -- start 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.327+0000 7f0d3979e740 1 -- start start 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b77510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 0x55a0a1b77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b47960 0x55a0a1b4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d3979e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0a1ab0740 con 0x55a0a1b750e0 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a0a1a4ea80 con 0x55a0a1b47960 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a0a1aac700 con 0x55a0a1b74d00 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d27fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 0x55a0a1b77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d27fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 0x55a0a1b77a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59602/0 (socket says 172.21.15.111:59602) 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d27fff640 1 -- 172.21.15.111:0/1315080040 learned_addr learned my addr 172.21.15.111:0/1315080040 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b47960 0x55a0a1b4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.328+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b77510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d27fff640 1 -- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 msgr2=0x55a0a1b77510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b77510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d27fff640 1 -- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b47960 msgr2=0x55a0a1b4b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b47960 0x55a0a1b4b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d27fff640 1 -- 172.21.15.111:0/1315080040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a1a79fb0 con 0x55a0a1b750e0 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b47960 0x55a0a1b4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 0x55a0a1b77a50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d18061670 tx=0x7f0d18059800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f634ca6c2effbe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d25ffb640 1 -- 172.21.15.111:0/1315080040 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18071020 con 0x55a0a1b750e0 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d25ffb640 1 -- 172.21.15.111:0/1315080040 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18061ba0 con 0x55a0a1b750e0 2024-05-27T00:40:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d25ffb640 1 -- 172.21.15.111:0/1315080040 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18021040 con 0x55a0a1b750e0 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 -- 172.21.15.111:0/1315080040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 msgr2=0x55a0a1b77a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 0x55a0a1b77a50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d18061670 tx=0x7f0d18059800 comp rx=0 tx=0).stop 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 -- 172.21.15.111:0/1315080040 shutdown_connections 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b47960 0x55a0a1b4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b750e0 0x55a0a1b77a50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1315080040 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b77510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 -- 172.21.15.111:0/1315080040 >> 172.21.15.111:0/1315080040 conn(0x55a0a1b49bd0 msgr2=0x55a0a1b92510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 -- 172.21.15.111:0/1315080040 shutdown_connections 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.329+0000 7f0d3979e740 1 -- 172.21.15.111:0/1315080040 wait complete. 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.330+0000 7f0d3979e740 1 -- start start 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.330+0000 7f0d3979e740 1 Processor -- start 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b750e0 0x55a0a1b71240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b73130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0a1b4cd80 con 0x55a0a1b72cb0 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a0a1a39c30 con 0x55a0a1b750e0 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a0a1b4d060 con 0x55a0a1b74d00 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d3979e740 10 client.0 _ll_register_callbacks cb 0x55a0a1a6b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b73130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b73130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59606/0 (socket says 172.21.15.111:59606) 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 -- 172.21.15.111:0/2203845193 learned_addr learned my addr 172.21.15.111:0/2203845193 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b750e0 0x55a0a1b71240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 -- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 msgr2=0x55a0a1b6d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b6d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 -- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b750e0 msgr2=0x55a0a1b71240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b750e0 0x55a0a1b71240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 -- 172.21.15.111:0/2203845193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a17c42d0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b73130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0d280015c0 tx=0x7f0d28001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a01ed0840da9cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.331+0000 7f0d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d0effd640 1 -- 172.21.15.111:0/2203845193 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2801a060 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d0effd640 1 -- 172.21.15.111:0/2203845193 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0d28015c90 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d18061ba0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d0effd640 1 -- 172.21.15.111:0/2203845193 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0d28011640 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 10 client.5622 fetch_fsmap learned FSMap version 17 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a0a188e6b0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d0effd640 1 -- 172.21.15.111:0/2203845193 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18061ba0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d0effd640 1 -- 172.21.15.111:0/2203845193 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f0d280117e0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 10 client.5622 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 10 client.5622 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 20 client.5622 populate_metadata read hostname 'smithi111' 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b93c90 0x55a0a1b78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 4 client.5622 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 --> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a1abde70 con 0x55a0a1b93c90 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b93c90 0x55a0a1b78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d0effd640 10 client.5622 ms_handle_connect on v2:172.21.15.111:6833/3118800442 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.332+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b93c90 0x55a0a1b78d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd192191c5a17c5 server_cookie=99fa7882ae14485b in_seq=0 out_seq=0 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d0effd640 1 -- 172.21.15.111:0/2203845193 <== mds.0 v2:172.21.15.111:6833/3118800442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0a1abde70 con 0x55a0a1b93c90 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d0effd640 10 client.5622 handle_command_reply: tid=0 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 client.5622 shutdown 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 msgr2=0x55a0a1b73130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b73130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0d280015c0 tx=0x7f0d28001a80 comp rx=0 tx=0).stop 2024-05-27T00:40:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 shutdown_connections 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b73130 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b750e0 0x55a0a1b71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b93c90 0x55a0a1b78d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2203845193 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b74d00 0x55a0a1b6d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d0effd640 0 client.5622 ms_handle_reset on v2:172.21.15.111:6833/3118800442 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 >> 172.21.15.111:0/2203845193 conn(0x55a0a1b47960 msgr2=0x55a0a1b900c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 shutdown_connections 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.333+0000 7f0d3979e740 1 -- 172.21.15.111:0/2203845193 wait complete. 2024-05-27T00:40:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.334+0000 7f0d3979e740 20 client.5622 trim_cache size 0 max 16384 2024-05-27T00:40:40.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.352+0000 7f0d3979e740 1 Processor -- start 2024-05-27T00:40:40.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.352+0000 7f0d3979e740 1 -- start start 2024-05-27T00:40:40.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b4e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 0x55a0a1b6d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1bb6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d3979e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0d301fff10 con 0x55a0a1b74d00 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f0d30143a20 con 0x55a0a1b72cb0 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f0d301ff8c0 con 0x55a0a1b750e0 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 0x55a0a1b6d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 0x55a0a1b6d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59610/0 (socket says 172.21.15.111:59610) 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 -- 172.21.15.111:0/1173096619 learned_addr learned my addr 172.21.15.111:0/1173096619 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 -- 172.21.15.111:0/1173096619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 msgr2=0x55a0a1bb6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1bb6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 -- 172.21.15.111:0/1173096619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b72cb0 msgr2=0x55a0a1b4e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b4e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 -- 172.21.15.111:0/1173096619 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a1b438f0 con 0x55a0a1b74d00 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d27fff640 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 0x55a0a1b6d3d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d18058cc0 tx=0x7f0d1801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489f149465854184 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d257fa640 1 -- 172.21.15.111:0/1173096619 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18071020 con 0x55a0a1b74d00 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d257fa640 1 -- 172.21.15.111:0/1173096619 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18055630 con 0x55a0a1b74d00 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.353+0000 7f0d257fa640 1 -- 172.21.15.111:0/1173096619 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18021040 con 0x55a0a1b74d00 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 -- 172.21.15.111:0/1173096619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 msgr2=0x55a0a1b6d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 0x55a0a1b6d3d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d18058cc0 tx=0x7f0d1801e000 comp rx=0 tx=0).stop 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 -- 172.21.15.111:0/1173096619 shutdown_connections 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1bb6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b74d00 0x55a0a1b6d3d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 --2- 172.21.15.111:0/1173096619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b72cb0 0x55a0a1b4e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 -- 172.21.15.111:0/1173096619 >> 172.21.15.111:0/1173096619 conn(0x55a0a1b47960 msgr2=0x55a0a1b43090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 -- 172.21.15.111:0/1173096619 shutdown_connections 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 -- 172.21.15.111:0/1173096619 wait complete. 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.354+0000 7f0d3979e740 1 -- start start 2024-05-27T00:40:40.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 Processor -- start 2024-05-27T00:40:40.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1bf7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b74d00 0x55a0a1c158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1c11850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0a1b93b10 con 0x55a0a1b72cb0 2024-05-27T00:40:40.361 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x55a0a1b780e0 con 0x55a0a1b74d00 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.355+0000 7f0d3979e740 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x55a0a1b94160 con 0x55a0a1b750e0 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d3979e740 10 client.0 _ll_register_callbacks cb 0x7f0d28007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1bf7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1bf7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59622/0 (socket says 172.21.15.111:59622) 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 -- 172.21.15.111:0/2548901232 learned_addr learned my addr 172.21.15.111:0/2548901232 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1c11850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 -- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 msgr2=0x55a0a1c11850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b74d00 0x55a0a1c158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1c11850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 -- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b74d00 msgr2=0x55a0a1c158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b74d00 0x55a0a1c158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 -- 172.21.15.111:0/2548901232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a1ba20b0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.362 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b74d00 0x55a0a1c158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d34f4d640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1c11850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d2ffff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1bf7720 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d200098c0 tx=0x7f0d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2684840af214c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d0effd640 1 -- 172.21.15.111:0/2548901232 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20037040 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d0effd640 1 -- 172.21.15.111:0/2548901232 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0d20032b40 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d18055630 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d0effd640 1 -- 172.21.15.111:0/2548901232 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0d20059050 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d3979e740 10 client.5628 fetch_fsmap learned FSMap version 17 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a0a1c123d0 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d0effd640 1 -- 172.21.15.111:0/2548901232 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18055630 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.356+0000 7f0d0effd640 1 -- 172.21.15.111:0/2548901232 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f0d2005c020 con 0x55a0a1b72cb0 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 10 client.5628 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T00:40:40.363 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 10 client.5628 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 20 client.5628 populate_metadata read hostname 'smithi111' 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b90ea0 0x55a0a1b91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 4 client.5628 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 --> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] -- command(tid 0: {"prefix": "status"}) -- 0x55a0a1ab81c0 con 0x55a0a1b90ea0 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b90ea0 0x55a0a1b91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d0effd640 10 client.5628 ms_handle_connect on v2:172.21.15.111:6833/3118800442 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d27fff640 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b90ea0 0x55a0a1b91280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f22968193051db4 server_cookie=dac8d66783c86153 in_seq=0 out_seq=0 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d0effd640 1 -- 172.21.15.111:0/2548901232 <== mds.0 v2:172.21.15.111:6833/3118800442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a0a1ab81c0 con 0x55a0a1b90ea0 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d0effd640 10 client.5628 handle_command_reply: tid=0 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 client.5628 shutdown 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 msgr2=0x55a0a1bf7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1bf7720 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d200098c0 tx=0x7f0d200046c0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 shutdown_connections 2024-05-27T00:40:40.364 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:6833/3118800442,v1:172.21.15.111:6835/3118800442] conn(0x55a0a1b90ea0 0x55a0a1b91280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a0a1b750e0 0x55a0a1c11850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a0a1b74d00 0x55a0a1c158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 --2- 172.21.15.111:0/2548901232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0a1b72cb0 0x55a0a1bf7720 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d0effd640 0 client.5628 ms_handle_reset on v2:172.21.15.111:6833/3118800442 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 >> 172.21.15.111:0/2548901232 conn(0x55a0a1b47960 msgr2=0x55a0a1b9a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 shutdown_connections 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.357+0000 7f0d3979e740 1 -- 172.21.15.111:0/2548901232 wait complete. 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d3979e740 20 client.5628 trim_cache size 0 max 16384 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-05-27T00:40:40.365 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "ef97d1dc-a8f7-4487-922a-ccc176f6927a", 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5418, 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 9.5292120709999999, 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 17, 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 39, 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 39, 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 16.642987046999998 2024-05-27T00:40:40.366 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-05-27T00:40:40.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0d1006be30 msgr2=0x7f0d1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0d1006be30 0x7f0d1006e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0d20034000 tx=0x7f0d2002c000 comp rx=0 tx=0).stop 2024-05-27T00:40:40.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 msgr2=0x7f0d300b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 0x7f0d300b3000 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800e9b0 tx=0x7f0d1800ee70 comp rx=0 tx=0).stop 2024-05-27T00:40:40.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 shutdown_connections 2024-05-27T00:40:40.367 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f0d1006be30 0x7f0d1006e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d3013f340 0x7f0d3006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3013ef60 0x7f0d300b3000 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 --2- 172.21.15.111:0/2162326010 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f0d300cce00 0x7f0d3013e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 >> 172.21.15.111:0/2162326010 conn(0x7f0d300b1c00 msgr2=0x7f0d3015ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 shutdown_connections 2024-05-27T00:40:40.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.358+0000 7f0d369b0640 1 -- 172.21.15.111:0/2162326010 wait complete. 2024-05-27T00:40:40.370 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T00:40:40.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.469+0000 7f57d1902640 1 Processor -- start 2024-05-27T00:40:40.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.469+0000 7f57d1902640 1 -- start start 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57d1902640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f57cc144cd0 0x7f57cc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57d1902640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57cc145af0 con 0x7f57cc145350 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57d1902640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57cc15fbd0 con 0x7f57cc0b5e40 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57d1902640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc160500 con 0x7f57cc144cd0 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57c27fc640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f57cc145350 0x7f57cc0b5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54176/0 (socket says 172.21.15.111:54176) 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57c27fc640 1 -- 172.21.15.111:0/2194918374 learned_addr learned my addr 172.21.15.111:0/2194918374 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T00:40:40.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 978253767 0 0) 0x7f57cc145af0 con 0x7f57cc145350 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57b8003520 con 0x7f57cc145350 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57caffd640 1 --2- 172.21.15.111:0/2194918374 >> v2:172.21.15.153:3300/0 conn(0x7f57cc144cd0 0x7f57cc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1448359817 0 0) 0x7f57b8003520 con 0x7f57cc145350 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 >> v2:172.21.15.153:3300/0 conn(0x7f57cc144cd0 msgr2=0x7f57cc06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 --2- 172.21.15.111:0/2194918374 >> v2:172.21.15.153:3300/0 conn(0x7f57cc144cd0 0x7f57cc06f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 >> v1:172.21.15.153:6789/0 conn(0x7f57cc0b5e40 legacy=0x7f57cc06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc15feb0 con 0x7f57cc145350 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7f57b4001670 con 0x7f57cc145350 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 >> v1:172.21.15.111:6789/0 conn(0x7f57cc145350 legacy=0x7f57cc0b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57b80039b0 0x7f57b8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57b80051c0 0x7f57b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.474 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc160500 con 0x7f57b8004800 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57b800c090 con 0x7f57b80039b0 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f57b800c290 con 0x7f57b80051c0 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57c27fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.470+0000 7f57caffd640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57b80039b0 0x7f57b8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57cb7fe640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57b80051c0 0x7f57b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57c27fc640 1 -- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57b80051c0 msgr2=0x7f57b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57c27fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57b80051c0 0x7f57b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57c27fc640 1 -- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57b80039b0 msgr2=0x7f57b8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57c27fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57b80039b0 0x7f57b8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57c27fc640 1 -- 172.21.15.111:0/2194918374 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8003520 con 0x7f57b8004800 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57cb7fe640 1 -- 172.21.15.111:0/2194918374 reap_dead start 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.471+0000 7f57c27fc640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57b8004c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c240 tx=0x7f57b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4873ef3645299d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4007d10 con 0x7f57b8004800 2024-05-27T00:40:40.475 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b40058d0 con 0x7f57b8004800 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57ca7fc640 1 -- 172.21.15.111:0/2194918374 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4005d00 con 0x7f57b8004800 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 -- 172.21.15.111:0/2194918374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 msgr2=0x7f57b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57b8004c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c240 tx=0x7f57b8005830 comp rx=0 tx=0).stop 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 -- 172.21.15.111:0/2194918374 shutdown_connections 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 --2- 172.21.15.111:0/2194918374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57b8004c80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 -- 172.21.15.111:0/2194918374 >> 172.21.15.111:0/2194918374 conn(0x7f57cc073130 msgr2=0x7f57cc0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 -- 172.21.15.111:0/2194918374 shutdown_connections 2024-05-27T00:40:40.476 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.472+0000 7f57d1902640 1 -- 172.21.15.111:0/2194918374 wait complete. 2024-05-27T00:40:40.478 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.473+0000 7f57d1902640 1 Processor -- start 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.473+0000 7f57d1902640 1 -- start start 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57d1902640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57cc15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57d1902640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57cc137ef0 0x7f57cc138370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57d1902640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57cc139d20 0x7f57cc13a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57d1902640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1450b0 con 0x7f57b8004800 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57d1902640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1457e0 con 0x7f57cc139d20 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57d1902640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc160990 con 0x7f57cc137ef0 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57caffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57cc15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57caffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57cc15ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60508/0 (socket says 172.21.15.111:60508) 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57caffd640 1 -- 172.21.15.111:0/1860601090 learned_addr learned my addr 172.21.15.111:0/1860601090 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57c27fc640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57cc137ef0 0x7f57cc138370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.474+0000 7f57cb7fe640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57cc139d20 0x7f57cc13a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.479 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57caffd640 1 -- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57cc137ef0 msgr2=0x7f57cc138370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57caffd640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57cc137ef0 0x7f57cc138370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57caffd640 1 -- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57cc139d20 msgr2=0x7f57cc13a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57caffd640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57cc139d20 0x7f57cc13a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57caffd640 1 -- 172.21.15.111:0/1860601090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1f1f10 con 0x7f57b8004800 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57c27fc640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57cc137ef0 0x7f57cc138370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57cb7fe640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57cc139d20 0x7f57cc13a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57caffd640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57cc15ade0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0119c0 tx=0x7f57bc011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00fb50 con 0x7f57b8004800 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc00fcf0 con 0x7f57b8004800 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00fea0 con 0x7f57b8004800 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc138970 con 0x7f57b8004800 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.475+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc141540 con 0x7f57b8004800 2024-05-27T00:40:40.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.476+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc068680 con 0x7f57b8004800 2024-05-27T00:40:40.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.476+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57bc012040 con 0x7f57b8004800 2024-05-27T00:40:40.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.477+0000 7f57c8ff9640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f57ac06be30 0x7f57ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:40:40.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.477+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f57bc08b5d0 con 0x7f57b8004800 2024-05-27T00:40:40.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.477+0000 7f57c27fc640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f57ac06be30 0x7f57ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:40:40.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.477+0000 7f57c27fc640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f57ac06be30 0x7f57ac06e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c240 tx=0x7f57b4035040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:40:40.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.481+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f57bc019030 con 0x7f57b8004800 2024-05-27T00:40:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.691+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57cc06dea0 con 0x7f57b8004800 2024-05-27T00:40:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.692+0000 7f57c8ff9640 1 -- 172.21.15.111:0/1860601090 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f57bc059e80 con 0x7f57b8004800 2024-05-27T00:40:40.693 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f57ac06be30 msgr2=0x7f57ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f57ac06be30 0x7f57ac06e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c240 tx=0x7f57b4035040 comp rx=0 tx=0).stop 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 msgr2=0x7f57cc15ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57cc15ade0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0119c0 tx=0x7f57bc011e80 comp rx=0 tx=0).stop 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 shutdown_connections 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f57ac06be30 0x7f57ac06e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57cc139d20 0x7f57cc13a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f57cc137ef0 0x7f57cc138370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 --2- 172.21.15.111:0/1860601090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57b8004800 0x7f57cc15ade0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 >> 172.21.15.111:0/1860601090 conn(0x7f57cc073130 msgr2=0x7f57cc0b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:40:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 shutdown_connections 2024-05-27T00:40:40.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40.694+0000 7f57d1902640 1 -- 172.21.15.111:0/1860601090 wait complete. 2024-05-27T00:40:40.705 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-27T00:40:40.705 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T00:40:40.706 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T00:40:40.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:40.706 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-05-27T00:40:40.722 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-05-27T00:40:40.723 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42679sec preferred_lft 42679sec 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether e2:c6:54:b8:65:47 brd ff:ff:ff:ff:ff:ff 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::50a1:3bff:fe88:d7a2/64 scope link 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.724 INFO:teuthology.orchestra.run.smithi111.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether fa:19:15:ca:eb:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::f819:15ff:feca:ebc8/64 scope link 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether e2:c6:54:b8:65:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::e0c6:54ff:feb8:6547/64 scope link 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-05-27T00:40:40.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:40.725 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T00:40:40.786 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-05-27T00:40:40.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:40.788 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-05-27T00:40:40.845 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-05-27T00:40:40.845 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T00:40:40.846 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T00:40:40.846 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T00:40:40.846 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T00:40:40.846 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T00:40:40.846 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-05-27T00:40:40.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T00:40:40.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T00:40:40.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T00:40:40.846 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-05-27T00:40:40.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T00:40:40.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:40:40.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:40.976 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-05-27T00:40:40.991 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-27T00:40:40.991 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T00:40:40.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T00:40:40.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:40.992 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-05-27T00:40:41.057 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:40:41.058 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T00:40:41.112 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T00:40:41.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T00:40:41.112 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T00:40:41.167 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T00:40:41.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:41.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:40:41.251 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:40:41.251 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T00:40:41.251 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:40:41.260 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:40:41.260 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T00:40:41.260 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:40:41.260 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:40:41.260 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:40:41.264 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T00:40:41.264 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:3300,172.21.15.153:3300,172.21.15.153:3301" 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790" 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.111:6789,172.21.15.153:6789,172.21.15.153:6790:/ 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=ef97d1dc-a8f7-4487-922a-ccc176f6927a" will pass to kernel 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T00:40:41.265 INFO:teuthology.orchestra.run.smithi111.stdout: You just mounted a file system that supports labels which does not 2024-05-27T00:40:41.266 INFO:teuthology.orchestra.run.smithi111.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T00:40:41.266 INFO:teuthology.orchestra.run.smithi111.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T00:40:41.266 INFO:teuthology.orchestra.run.smithi111.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T00:40:41.267 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T00:40:41.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:41.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T00:40:41.333 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T00:40:41.423 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T00:40:41.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:40:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T00:40:41.433 INFO:teuthology.orchestra.run.smithi111.stdout:client5637 2024-05-27T00:40:41.434 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:40:41.434 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T00:40:41.451 DEBUG:teuthology.orchestra.run.smithi111:> mktemp -d --suffix=xfstests-dev 2024-05-27T00:40:41.507 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.pEH4Qq3ph1xfstests-dev 2024-05-27T00:40:41.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:41.509 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.pEH4Qq3ph1xfstests-dev) 2024-05-27T00:40:41.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:41.574 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.pEH4Qq3ph1xfstests-dev) 2024-05-27T00:40:41.632 INFO:teuthology.orchestra.run.smithi111.stderr:Cloning into '/tmp/tmp.pEH4Qq3ph1xfstests-dev'... 2024-05-27T00:40:42.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:42.419 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-05-27T00:40:42.479 DEBUG:teuthology.orchestra.run.smithi111:> mktemp -d --suffix=test 2024-05-27T00:40:42.535 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.SabVhXOPketest 2024-05-27T00:40:42.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:40:42.535 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-05-27T00:40:42.593 DEBUG:teuthology.orchestra.run.smithi111:> mktemp -d --suffix=scratch 2024-05-27T00:40:42.648 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:40:42.648 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-05-27T00:40:42.648 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-05-27T00:40:43.227 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:03:44 ago on Mon 27 May 2024 12:36:59 AM UTC. 2024-05-27T00:40:43.306 INFO:teuthology.orchestra.run.smithi111.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T00:40:43.307 INFO:teuthology.orchestra.run.smithi111.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T00:40:43.308 INFO:teuthology.orchestra.run.smithi111.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-05-27T00:40:43.309 INFO:teuthology.orchestra.run.smithi111.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-05-27T00:40:43.309 INFO:teuthology.orchestra.run.smithi111.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-05-27T00:40:43.311 INFO:teuthology.orchestra.run.smithi111.stdout:Package e2fsprogs-1.46.5-5.el9.x86_64 is already installed. 2024-05-27T00:40:43.313 INFO:teuthology.orchestra.run.smithi111.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-05-27T00:40:43.313 INFO:teuthology.orchestra.run.smithi111.stdout:Package gcc-11.4.1-3.el9.x86_64 is already installed. 2024-05-27T00:40:43.315 INFO:teuthology.orchestra.run.smithi111.stdout:Package git-2.43.0-1.el9.x86_64 is already installed. 2024-05-27T00:40:43.317 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T00:40:43.318 INFO:teuthology.orchestra.run.smithi111.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-05-27T00:40:43.320 INFO:teuthology.orchestra.run.smithi111.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-05-27T00:40:43.320 INFO:teuthology.orchestra.run.smithi111.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T00:40:43.321 INFO:teuthology.orchestra.run.smithi111.stdout:Package lvm2-9:2.03.23-2.el9.x86_64 is already installed. 2024-05-27T00:40:43.322 INFO:teuthology.orchestra.run.smithi111.stdout:Package make-1:4.3-8.el9.x86_64 is already installed. 2024-05-27T00:40:43.323 INFO:teuthology.orchestra.run.smithi111.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-05-27T00:40:43.324 INFO:teuthology.orchestra.run.smithi111.stdout:Package python3-3.9.18-3.el9.x86_64 is already installed. 2024-05-27T00:40:43.325 INFO:teuthology.orchestra.run.smithi111.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-05-27T00:40:43.325 INFO:teuthology.orchestra.run.smithi111.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-05-27T00:40:43.328 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T00:40:43.373 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout: gdbm-devel x86_64 1:1.23-1.el9 crb 81 k 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-05-27T00:40:43.375 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-devel x86_64 5.14.0-447.el9 appstream 16 M 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: libcap-devel x86_64 2.48-9.el9 appstream 41 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: udftools x86_64 2.2-5.el9 appstream 149 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: gdbm-libs x86_64 1:1.23-1.el9 baseos 56 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: gdbm x86_64 1:1.23-1.el9 crb 153 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: libnbd x86_64 1.20.0-1.el9 appstream 170 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: rmt x86_64 2:1.6-6.el9 baseos 46 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-05-27T00:40:43.376 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout:Install 17 Packages 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 1 Package 2024-05-27T00:40:43.377 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:40:43.378 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 24 M 2024-05-27T00:40:43.378 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-05-27T00:40:43.991 INFO:teuthology.orchestra.run.smithi111.stdout:(1/18): fio-engine-http-3.35-1.el9.x86_64.rpm 207 kB/s | 18 kB 00:00 2024-05-27T00:40:44.016 INFO:teuthology.orchestra.run.smithi111.stdout:(2/18): rmt-1.6-6.el9.x86_64.rpm 409 kB/s | 46 kB 00:00 2024-05-27T00:40:44.046 INFO:teuthology.orchestra.run.smithi111.stdout:(3/18): fio-engine-libaio-3.35-1.el9.x86_64.rpm 282 kB/s | 16 kB 00:00 2024-05-27T00:40:44.066 INFO:teuthology.orchestra.run.smithi111.stdout:(4/18): fio-engine-nbd-3.35-1.el9.x86_64.rpm 290 kB/s | 15 kB 00:00 2024-05-27T00:40:44.092 INFO:teuthology.orchestra.run.smithi111.stdout:(5/18): fio-engine-rados-3.35-1.el9.x86_64.rpm 347 kB/s | 16 kB 00:00 2024-05-27T00:40:44.142 INFO:teuthology.orchestra.run.smithi111.stdout:(6/18): fio-engine-rdma-3.35-1.el9.x86_64.rpm 413 kB/s | 21 kB 00:00 2024-05-27T00:40:44.167 INFO:teuthology.orchestra.run.smithi111.stdout:(7/18): fio-engine-rbd-3.35-1.el9.x86_64.rpm 161 kB/s | 16 kB 00:00 2024-05-27T00:40:44.318 INFO:teuthology.orchestra.run.smithi111.stdout:(8/18): fio-3.35-1.el9.x86_64.rpm 15 MB/s | 6.2 MB 00:00 2024-05-27T00:40:44.343 INFO:teuthology.orchestra.run.smithi111.stdout:(9/18): libcap-devel-2.48-9.el9.x86_64.rpm 231 kB/s | 41 kB 00:00 2024-05-27T00:40:44.370 INFO:teuthology.orchestra.run.smithi111.stdout:(10/18): libnbd-1.20.0-1.el9.x86_64.rpm 3.2 MB/s | 170 kB 00:00 2024-05-27T00:40:44.454 INFO:teuthology.orchestra.run.smithi111.stdout:(11/18): udftools-2.2-5.el9.x86_64.rpm 1.7 MB/s | 149 kB 00:00 2024-05-27T00:40:44.554 INFO:teuthology.orchestra.run.smithi111.stdout:(12/18): sqlite-3.34.1-7.el9.x86_64.rpm 3.5 MB/s | 748 kB 00:00 2024-05-27T00:40:44.613 INFO:teuthology.orchestra.run.smithi111.stdout:(13/18): gdbm-1.23-1.el9.x86_64.rpm 964 kB/s | 153 kB 00:00 2024-05-27T00:40:44.664 INFO:teuthology.orchestra.run.smithi111.stdout:(14/18): gdbm-devel-1.23-1.el9.x86_64.rpm 739 kB/s | 81 kB 00:00 2024-05-27T00:40:45.155 INFO:teuthology.orchestra.run.smithi111.stdout:(15/18): kernel-devel-5.14.0-447.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-05-27T00:40:45.181 INFO:teuthology.orchestra.run.smithi111.stdout:(16/18): dump-0.4-0.51.b47.el9.x86_64.rpm 155 kB/s | 88 kB 00:00 2024-05-27T00:40:45.206 INFO:teuthology.orchestra.run.smithi111.stdout:(17/18): indent-2.2.13-5.el9.x86_64.rpm 331 kB/s | 179 kB 00:00 2024-05-27T00:40:45.232 INFO:teuthology.orchestra.run.smithi111.stdout:(18/18): gdbm-libs-1.23-1.el9.x86_64.rpm 742 kB/s | 56 kB 00:00 2024-05-27T00:40:45.232 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-05-27T00:40:45.232 INFO:teuthology.orchestra.run.smithi111.stdout:Total 13 MB/s | 24 MB 00:01 2024-05-27T00:40:45.579 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-05-27T00:40:45.637 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-05-27T00:40:45.637 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-05-27T00:40:46.034 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:40:46.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:40:46.249 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-05-27T00:40:46.249 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-05-27T00:40:47.281 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-05-27T00:40:47.347 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : gdbm-libs-1:1.23-1.el9.x86_64 1/19 2024-05-27T00:40:47.386 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gdbm-1:1.23-1.el9.x86_64 2/19 2024-05-27T00:40:47.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libnbd-1.20.0-1.el9.x86_64 3/19 2024-05-27T00:40:47.440 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 4/19 2024-05-27T00:40:47.465 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 5/19 2024-05-27T00:40:47.490 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 6/19 2024-05-27T00:40:47.516 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 7/19 2024-05-27T00:40:47.541 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 8/19 2024-05-27T00:40:47.694 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 9/19 2024-05-27T00:40:47.747 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-3.35-1.el9.x86_64 10/19 2024-05-27T00:40:47.783 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rmt-2:1.6-6.el9.x86_64 11/19 2024-05-27T00:40:47.814 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 12/19 2024-05-27T00:40:47.856 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gdbm-devel-1:1.23-1.el9.x86_64 13/19 2024-05-27T00:40:47.900 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : indent-2.2.13-5.el9.x86_64 14/19 2024-05-27T00:40:47.940 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : udftools-2.2-5.el9.x86_64 15/19 2024-05-27T00:40:47.988 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 16/19 2024-05-27T00:40:57.325 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 17/19 2024-05-27T00:40:57.590 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-devel-5.14.0-447.el9.x86_64 18/19 2024-05-27T00:41:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-devel-5.14.0-447.el9.x86_64 18/19 2024-05-27T00:41:08.368 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : gdbm-libs-1:1.19-4.el9.x86_64 19/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: gdbm-libs-1:1.19-4.el9.x86_64 19/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 1/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-3.35-1.el9.x86_64 2/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 3/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 4/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 5/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 6/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 7/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 8/19 2024-05-27T00:41:10.369 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-devel-5.14.0-447.el9.x86_64 9/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 10/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libnbd-1.20.0-1.el9.x86_64 11/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 12/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : udftools-2.2-5.el9.x86_64 13/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gdbm-1:1.23-1.el9.x86_64 14/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gdbm-devel-1:1.23-1.el9.x86_64 15/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/19 2024-05-27T00:41:10.370 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gdbm-libs-1:1.23-1.el9.x86_64 18/19 2024-05-27T00:41:10.823 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gdbm-libs-1:1.19-4.el9.x86_64 19/19 2024-05-27T00:41:10.823 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:41:10.823 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-05-27T00:41:10.823 INFO:teuthology.orchestra.run.smithi111.stdout: gdbm-libs-1:1.23-1.el9.x86_64 2024-05-27T00:41:10.823 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-05-27T00:41:10.823 INFO:teuthology.orchestra.run.smithi111.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: gdbm-1:1.23-1.el9.x86_64 gdbm-devel-1:1.23-1.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-447.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.20.0-1.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: udftools-2.2-5.el9.x86_64 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:41:10.824 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-05-27T00:41:11.200 DEBUG:teuthology.orchestra.run.smithi111:> sudo useradd -m fsgqa 2024-05-27T00:41:11.226 INFO:teuthology.orchestra.run.smithi111.stderr:useradd: user 'fsgqa' already exists 2024-05-27T00:41:11.228 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-05-27T00:41:11.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo groupadd fsgqa 2024-05-27T00:41:11.293 INFO:teuthology.orchestra.run.smithi111.stderr:groupadd: group 'fsgqa' already exists 2024-05-27T00:41:11.294 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-05-27T00:41:11.295 DEBUG:teuthology.orchestra.run.smithi111:> sudo useradd fsgqa2 2024-05-27T00:41:12.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo useradd 123456-fsgqa 2024-05-27T00:41:12.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-05-27T00:41:13.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.000+0000 7fc5c09f9640 1 Processor -- start 2024-05-27T00:41:13.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.000+0000 7fc5c09f9640 1 -- start start 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5c09f9640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc5bc13ef60 0x7fc5bc0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5c09f9640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5bc13f340 0x7fc5bc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5c09f9640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5bc142650 con 0x7fc5bc0cce00 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5c09f9640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fc5bc0cd270 con 0x7fc5bc13f340 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5c09f9640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fc5bc0cf210 con 0x7fc5bc13ef60 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5bc13f340 0x7fc5bc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5bc13f340 0x7fc5bc0cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34942/0 (socket says 172.21.15.111:34942) 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/986231636 learned_addr learned my addr 172.21.15.111:0/986231636 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5bad9d640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc5bc0cce00 0x7fc5bc141920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:34186/0 (socket says 172.21.15.111:34186) 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/986231636 >> v2:172.21.15.153:3300/0 conn(0x7fc5bc13ef60 msgr2=0x7fc5bc0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5ba59c640 1 --2- 172.21.15.111:0/986231636 >> v2:172.21.15.153:3300/0 conn(0x7fc5bc13ef60 0x7fc5bc0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 --2- 172.21.15.111:0/986231636 >> v2:172.21.15.153:3300/0 conn(0x7fc5bc13ef60 0x7fc5bc0cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/986231636 >> v1:172.21.15.111:6789/0 conn(0x7fc5bc0cce00 legacy=0x7fc5bc141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b959a640 1 -- 172.21.15.111:0/986231636 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743420012 0 0) 0x7fc5bc142650 con 0x7fc5bc0cce00 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/986231636 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc0cebc0 con 0x7fc5bc13f340 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.001+0000 7fc5ba59c640 1 --2- 172.21.15.111:0/986231636 >> v2:172.21.15.153:3300/0 conn(0x7fc5bc13ef60 0x7fc5bc0cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.002+0000 7fc5b9d9b640 1 --2- 172.21.15.111:0/986231636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f340 0x7fc5bc0cc8c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0742a0 tx=0x7fc5a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fc7b599a1e721 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.002+0000 7fc5b959a640 1 -- 172.21.15.111:0/986231636 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4037070 con 0x7fc5bc13f340 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.002+0000 7fc5b959a640 1 -- 172.21.15.111:0/986231636 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4032440 con 0x7fc5bc13f340 2024-05-27T00:41:13.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.002+0000 7fc5b959a640 1 -- 172.21.15.111:0/986231636 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4032810 con 0x7fc5bc13f340 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.002+0000 7fc5c09f9640 1 -- 172.21.15.111:0/986231636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f340 msgr2=0x7fc5bc0cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.002+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/986231636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f340 0x7fc5bc0cc8c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0742a0 tx=0x7fc5a402a270 comp rx=0 tx=0).stop 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.003+0000 7fc5c09f9640 1 -- 172.21.15.111:0/986231636 shutdown_connections 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.003+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/986231636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13f340 0x7fc5bc0cc8c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.003+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/986231636 >> v2:172.21.15.153:3300/0 conn(0x7fc5bc13ef60 0x7fc5bc0cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.003+0000 7fc5c09f9640 1 -- 172.21.15.111:0/986231636 >> 172.21.15.111:0/986231636 conn(0x7fc5bc0b1c00 msgr2=0x7fc5bc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:41:13.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.003+0000 7fc5c09f9640 1 -- 172.21.15.111:0/986231636 shutdown_connections 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.003+0000 7fc5c09f9640 1 -- 172.21.15.111:0/986231636 wait complete. 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.004+0000 7fc5c09f9640 1 Processor -- start 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.004+0000 7fc5c09f9640 1 -- start start 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5c09f9640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5bc0cce00 0x7fc5bc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5c09f9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 0x7fc5bc144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5c09f9640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5bc13f340 0x7fc5bc156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5c09f9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc0ce890 con 0x7fc5bc13ef60 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5c09f9640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc0cfbe0 con 0x7fc5bc13f340 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5c09f9640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc142030 con 0x7fc5bc0cce00 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 0x7fc5bc144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 0x7fc5bc144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34946/0 (socket says 172.21.15.111:34946) 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/597573479 learned_addr learned my addr 172.21.15.111:0/597573479 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:41:13.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5ba59c640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5bc0cce00 0x7fc5bc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5bad9d640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5bc13f340 0x7fc5bc156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5bc0cce00 msgr2=0x7fc5bc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5bc0cce00 0x7fc5bc1558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5bc13f340 msgr2=0x7fc5bc156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5bc13f340 0x7fc5bc156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 -- 172.21.15.111:0/597573479 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc155e20 con 0x7fc5bc13ef60 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.005+0000 7fc5b9d9b640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 0x7fc5bc144000 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc5a402a2b0 tx=0x7fc5a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.006+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4037040 con 0x7fc5bc13ef60 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.006+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4042070 con 0x7fc5bc13ef60 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.006+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc1449a0 con 0x7fc5bc13ef60 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.006+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4030bb0 con 0x7fc5bc13ef60 2024-05-27T00:41:13.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.006+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc144f60 con 0x7fc5bc13ef60 2024-05-27T00:41:13.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.007+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5bc068680 con 0x7fc5bc13ef60 2024-05-27T00:41:13.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.007+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5a4047460 con 0x7fc5bc13ef60 2024-05-27T00:41:13.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.008+0000 7fc5b2ffd640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc59c06be30 0x7fc59c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.008+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fc5a40abc40 con 0x7fc5bc13ef60 2024-05-27T00:41:13.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.008+0000 7fc5ba59c640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc59c06be30 0x7fc59c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.008+0000 7fc5ba59c640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc59c06be30 0x7fc59c06e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00a990 tx=0x7fc5ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:41:13.013 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.013+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc5bc155e20 con 0x7fc5bc13ef60 2024-05-27T00:41:13.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.256+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc5bc0d06c0 con 0x7fc5bc13ef60 2024-05-27T00:41:13.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.257+0000 7fc5b2ffd640 1 -- 172.21.15.111:0/597573479 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7fc5a4045360 con 0x7fc5bc13ef60 2024-05-27T00:41:13.258 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:41:13.258 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":1,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","modified":"2024-05-27T00:38:17.301055Z","created":"2024-05-27T00:38:17.301055Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3301","nonce":0},{"type":"v1","addr":"172.21.15.153:6790","nonce":0}]},"addr":"172.21.15.153:6790/0","public_addr":"172.21.15.153:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-27T00:41:13.258 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 1 2024-05-27T00:41:13.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc59c06be30 msgr2=0x7fc59c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc59c06be30 0x7fc59c06e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00a990 tx=0x7fc5ac008040 comp rx=0 tx=0).stop 2024-05-27T00:41:13.260 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 msgr2=0x7fc5bc144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 0x7fc5bc144000 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc5a402a2b0 tx=0x7fc5a40046c0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 shutdown_connections 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fc59c06be30 0x7fc59c06e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5bc13f340 0x7fc5bc156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5bc13ef60 0x7fc5bc144000 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 --2- 172.21.15.111:0/597573479 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fc5bc0cce00 0x7fc5bc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 >> 172.21.15.111:0/597573479 conn(0x7fc5bc0b1c00 msgr2=0x7fc5bc0af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 shutdown_connections 2024-05-27T00:41:13.261 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.259+0000 7fc5c09f9640 1 -- 172.21.15.111:0/597573479 wait complete. 2024-05-27T00:41:13.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-05-27T00:41:13.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.401+0000 7fb48cd9e640 1 Processor -- start 2024-05-27T00:41:13.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.401+0000 7fb48cd9e640 1 -- start start 2024-05-27T00:41:13.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.401+0000 7fb48cd9e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 0x7fb4880cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb48cd9e640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb48813e050 0x7fb4880ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb48cd9e640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4881406a0 con 0x7fb48815b7e0 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb48cd9e640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fb488140960 con 0x7fb48813e050 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb48cd9e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb488140c40 con 0x7fb48813dc70 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 0x7fb4880cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 0x7fb4880cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:39336/0 (socket says 172.21.15.111:39336) 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb486ffd640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb48813e050 0x7fb4880ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 -- 172.21.15.111:0/843273121 learned_addr learned my addr 172.21.15.111:0/843273121 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 -- 172.21.15.111:0/843273121 >> v2:172.21.15.153:3300/0 conn(0x7fb48813e050 msgr2=0x7fb4880ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 --2- 172.21.15.111:0/843273121 >> v2:172.21.15.153:3300/0 conn(0x7fb48813e050 0x7fb4880ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 -- 172.21.15.111:0/843273121 >> v1:172.21.15.153:6789/0 conn(0x7fb48815b7e0 legacy=0x7fb48815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb4877fe640 1 -- 172.21.15.111:0/843273121 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb488140f20 con 0x7fb48813dc70 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.402+0000 7fb486ffd640 1 --2- 172.21.15.111:0/843273121 >> v2:172.21.15.153:3300/0 conn(0x7fb48813e050 0x7fb4880ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.403+0000 7fb4877fe640 1 --2- 172.21.15.111:0/843273121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 0x7fb4880cc720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb478009870 tx=0x7fb47802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b780d822f9db2c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.403+0000 7fb4867fc640 1 -- 172.21.15.111:0/843273121 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478037070 con 0x7fb48813dc70 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.403+0000 7fb4867fc640 1 -- 172.21.15.111:0/843273121 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb478032440 con 0x7fb48813dc70 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.403+0000 7fb4867fc640 1 -- 172.21.15.111:0/843273121 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478032810 con 0x7fb48813dc70 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 -- 172.21.15.111:0/843273121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 msgr2=0x7fb4880cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/843273121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 0x7fb4880cc720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb478009870 tx=0x7fb47802a270 comp rx=0 tx=0).stop 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 -- 172.21.15.111:0/843273121 shutdown_connections 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/843273121 >> v2:172.21.15.153:3300/0 conn(0x7fb48813e050 0x7fb4880ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/843273121 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813dc70 0x7fb4880cc720 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 -- 172.21.15.111:0/843273121 >> 172.21.15.111:0/843273121 conn(0x7fb48806d300 msgr2=0x7fb48806d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:41:13.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 -- 172.21.15.111:0/843273121 shutdown_connections 2024-05-27T00:41:13.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.404+0000 7fb48cd9e640 1 -- 172.21.15.111:0/843273121 wait complete. 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.405+0000 7fb48cd9e640 1 Processor -- start 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.405+0000 7fb48cd9e640 1 -- start start 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb48cd9e640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48813dc70 0x7fb488131ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb48cd9e640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813e050 0x7fb488136130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb48cd9e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 0x7fb488136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb48cd9e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb488071aa0 con 0x7fb48815b7e0 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb48cd9e640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb488071920 con 0x7fb48813dc70 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb48cd9e640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb488071c20 con 0x7fb48813e050 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 0x7fb488136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 0x7fb488136670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34952/0 (socket says 172.21.15.111:34952) 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 -- 172.21.15.111:0/3250408988 learned_addr learned my addr 172.21.15.111:0/3250408988 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb486ffd640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813e050 0x7fb488136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb4877fe640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48813dc70 0x7fb488131ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 -- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813e050 msgr2=0x7fb488136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813e050 0x7fb488136130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 -- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48813dc70 msgr2=0x7fb488131ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48813dc70 0x7fb488131ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.406+0000 7fb487fff640 1 -- 172.21.15.111:0/3250408988 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48813bb80 con 0x7fb48815b7e0 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.407+0000 7fb487fff640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 0x7fb488136670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c730 tx=0x7fb48000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.407+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480003b70 con 0x7fb48815b7e0 2024-05-27T00:41:13.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.407+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb480003d30 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.407+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48813be60 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.407+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480005310 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.407+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881394b0 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.408+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4880683f0 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.408+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4800054b0 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.409+0000 7fb46ffff640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb47406be30 0x7fb47406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.409+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fb48008ae80 con 0x7fb48815b7e0 2024-05-27T00:41:13.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.409+0000 7fb4877fe640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb47406be30 0x7fb47406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T00:41:13.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.409+0000 7fb4877fe640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb47406be30 0x7fb47406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb478034000 tx=0x7fb47802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T00:41:13.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.413+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb480058fb0 con 0x7fb48815b7e0 2024-05-27T00:41:13.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.656+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) -- 0x7fb4880af690 con 0x7fb48815b7e0 2024-05-27T00:41:13.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.656+0000 7fb46ffff640 1 -- 172.21.15.111:0/3250408988 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v10) ==== 92+0+63 (secure 0 0 0) 0x7fb48005d2b0 con 0x7fb48815b7e0 2024-05-27T00:41:13.657 INFO:teuthology.orchestra.run.smithi111.stdout:[client.admin] 2024-05-27T00:41:13.657 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== 2024-05-27T00:41:13.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb47406be30 msgr2=0x7fb47406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb47406be30 0x7fb47406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb478034000 tx=0x7fb47802e000 comp rx=0 tx=0).stop 2024-05-27T00:41:13.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 msgr2=0x7fb488136670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 0x7fb488136670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c730 tx=0x7fb48000cbf0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 shutdown_connections 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb47406be30 0x7fb47406e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb48815b7e0 0x7fb488136670 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb48813e050 0x7fb488136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 --2- 172.21.15.111:0/3250408988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb48813dc70 0x7fb488131ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.658+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 >> 172.21.15.111:0/3250408988 conn(0x7fb48806d300 msgr2=0x7fb48815dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.659+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 shutdown_connections 2024-05-27T00:41:13.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T00:41:13.659+0000 7fb48cd9e640 1 -- 172.21.15.111:0/3250408988 wait complete. 2024-05-27T00:41:13.671 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:41:13.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/tmp/tmp.pEH4Qq3ph1xfstests-dev/local.config 2024-05-27T00:41:13.696 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.111:6789:/test export TEST_DIR=/tmp/tmp.SabVhXOPketest export SCRATCH_DEV=172.21.15.111:6789:/scratch export SCRATCH_MNT=/tmp/tmp.dDEeBnibCbscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg==" 2024-05-27T00:41:13.696 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T00:41:13.696 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/tmp/tmp.pEH4Qq3ph1xfstests-dev/ceph.exclude 2024-05-27T00:41:13.762 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo make) 2024-05-27T00:41:13.840 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize -cfi 2024-05-27T00:41:14.221 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-27T00:41:14.221 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './config.guess' 2024-05-27T00:41:14.224 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './config.sub' 2024-05-27T00:41:14.228 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './install-sh' 2024-05-27T00:41:14.232 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './ltmain.sh' 2024-05-27T00:41:14.254 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-27T00:41:14.254 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-27T00:41:14.258 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-27T00:41:14.263 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-27T00:41:14.268 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-27T00:41:14.272 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-27T00:41:14.278 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-27T00:41:14.279 INFO:teuthology.orchestra.run.smithi111.stdout:cp include/install-sh . 2024-05-27T00:41:14.280 INFO:teuthology.orchestra.run.smithi111.stdout:aclocal -I m4 2024-05-27T00:41:15.268 INFO:teuthology.orchestra.run.smithi111.stdout:autoheader 2024-05-27T00:41:15.557 INFO:teuthology.orchestra.run.smithi111.stdout:autoconf 2024-05-27T00:41:15.726 INFO:teuthology.orchestra.run.smithi111.stdout:./configure \ 2024-05-27T00:41:15.727 INFO:teuthology.orchestra.run.smithi111.stdout: --libexecdir=/usr/lib \ 2024-05-27T00:41:15.727 INFO:teuthology.orchestra.run.smithi111.stdout: --exec_prefix=/var/lib 2024-05-27T00:41:16.134 INFO:teuthology.orchestra.run.smithi111.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-27T00:41:16.134 INFO:teuthology.orchestra.run.smithi111.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-27T00:41:16.135 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to print strings... printf 2024-05-27T00:41:16.135 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc... gcc 2024-05-27T00:41:16.190 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:41:16.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:41:16.287 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether the C compiler works... yes 2024-05-27T00:41:16.287 INFO:teuthology.orchestra.run.smithi111.stdout:checking for C compiler default output file name... a.out 2024-05-27T00:41:16.314 INFO:teuthology.orchestra.run.smithi111.stdout:checking for suffix of executables... 2024-05-27T00:41:16.376 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether we are cross compiling... no 2024-05-27T00:41:16.393 INFO:teuthology.orchestra.run.smithi111.stdout:checking for suffix of object files... o 2024-05-27T00:41:16.409 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether we are using the GNU C compiler... yes 2024-05-27T00:41:16.424 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether gcc accepts -g... yes 2024-05-27T00:41:16.469 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-27T00:41:16.471 INFO:teuthology.orchestra.run.smithi111.stdout:checking for a sed that does not truncate output... /bin/sed 2024-05-27T00:41:16.473 INFO:teuthology.orchestra.run.smithi111.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-05-27T00:41:16.474 INFO:teuthology.orchestra.run.smithi111.stdout:checking for egrep... /bin/grep -E 2024-05-27T00:41:16.476 INFO:teuthology.orchestra.run.smithi111.stdout:checking for fgrep... /bin/grep -F 2024-05-27T00:41:16.479 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ld used by gcc... /bin/ld 2024-05-27T00:41:16.480 INFO:teuthology.orchestra.run.smithi111.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-05-27T00:41:16.491 INFO:teuthology.orchestra.run.smithi111.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-05-27T00:41:17.390 INFO:teuthology.orchestra.run.smithi111.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-05-27T00:41:17.390 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether ln -s works... yes 2024-05-27T00:41:17.399 INFO:teuthology.orchestra.run.smithi111.stdout:checking the maximum length of command line arguments... 1572864 2024-05-27T00:41:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-27T00:41:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-27T00:41:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:checking for /bin/ld option to reload object files... -r 2024-05-27T00:41:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:checking for objdump... objdump 2024-05-27T00:41:17.407 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to recognize dependent libraries... pass_all 2024-05-27T00:41:17.408 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dlltool... no 2024-05-27T00:41:17.408 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-27T00:41:17.408 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ar... ar 2024-05-27T00:41:17.457 INFO:teuthology.orchestra.run.smithi111.stdout:checking for archiver @FILE support... @ 2024-05-27T00:41:17.457 INFO:teuthology.orchestra.run.smithi111.stdout:checking for strip... strip 2024-05-27T00:41:17.457 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ranlib... ranlib 2024-05-27T00:41:17.457 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gawk... gawk 2024-05-27T00:41:17.523 INFO:teuthology.orchestra.run.smithi111.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-05-27T00:41:17.525 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sysroot... no 2024-05-27T00:41:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:checking for a working dd... /bin/dd 2024-05-27T00:41:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-05-27T00:41:17.543 INFO:teuthology.orchestra.run.smithi111.stdout:checking for mt... no 2024-05-27T00:41:17.545 INFO:teuthology.orchestra.run.smithi111.stdout:checking if : is a manifest tool... no 2024-05-27T00:41:17.605 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to run the C preprocessor... gcc -E 2024-05-27T00:41:17.755 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ANSI C header files... yes 2024-05-27T00:41:17.776 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/types.h... yes 2024-05-27T00:41:17.801 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/stat.h... yes 2024-05-27T00:41:17.825 INFO:teuthology.orchestra.run.smithi111.stdout:checking for stdlib.h... yes 2024-05-27T00:41:17.850 INFO:teuthology.orchestra.run.smithi111.stdout:checking for string.h... yes 2024-05-27T00:41:17.875 INFO:teuthology.orchestra.run.smithi111.stdout:checking for memory.h... yes 2024-05-27T00:41:17.899 INFO:teuthology.orchestra.run.smithi111.stdout:checking for strings.h... yes 2024-05-27T00:41:17.927 INFO:teuthology.orchestra.run.smithi111.stdout:checking for inttypes.h... yes 2024-05-27T00:41:17.955 INFO:teuthology.orchestra.run.smithi111.stdout:checking for stdint.h... yes 2024-05-27T00:41:17.987 INFO:teuthology.orchestra.run.smithi111.stdout:checking for unistd.h... yes 2024-05-27T00:41:18.012 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dlfcn.h... yes 2024-05-27T00:41:18.018 INFO:teuthology.orchestra.run.smithi111.stdout:checking for objdir... .libs 2024-05-27T00:41:18.078 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-27T00:41:18.078 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-27T00:41:18.093 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-27T00:41:18.119 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc static flag -static works... no 2024-05-27T00:41:18.140 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc supports -c -o file.o... yes 2024-05-27T00:41:18.140 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-27T00:41:18.146 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-27T00:41:18.172 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether -lc should be explicitly linked in... no 2024-05-27T00:41:18.221 INFO:teuthology.orchestra.run.smithi111.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-27T00:41:18.221 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to hardcode library paths into programs... immediate 2024-05-27T00:41:18.222 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether stripping libraries is possible... yes 2024-05-27T00:41:18.222 INFO:teuthology.orchestra.run.smithi111.stdout:checking if libtool supports shared libraries... yes 2024-05-27T00:41:18.222 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether to build shared libraries... yes 2024-05-27T00:41:18.222 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether to build static libraries... yes 2024-05-27T00:41:18.228 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc... (cached) gcc 2024-05-27T00:41:18.243 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-27T00:41:18.244 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-27T00:41:18.244 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-27T00:41:18.244 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gmake... /usr/bin/gmake 2024-05-27T00:41:18.244 INFO:teuthology.orchestra.run.smithi111.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-05-27T00:41:18.245 INFO:teuthology.orchestra.run.smithi111.stdout:checking for tar... /bin/tar 2024-05-27T00:41:18.245 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gzip... /bin/gzip 2024-05-27T00:41:18.265 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether gcc -MM is supported... yes 2024-05-27T00:41:18.265 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sort... /bin/sort 2024-05-27T00:41:18.265 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether ln -s works... yes 2024-05-27T00:41:18.266 INFO:teuthology.orchestra.run.smithi111.stdout:checking for rpm... /bin/rpm 2024-05-27T00:41:18.269 INFO:teuthology.orchestra.run.smithi111.stdout:checking for rpmbuild... no 2024-05-27T00:41:18.293 INFO:teuthology.orchestra.run.smithi111.stdout:checking assert.h usability... yes 2024-05-27T00:41:18.301 INFO:teuthology.orchestra.run.smithi111.stdout:checking assert.h presence... yes 2024-05-27T00:41:18.302 INFO:teuthology.orchestra.run.smithi111.stdout:checking for assert.h... yes 2024-05-27T00:41:18.328 INFO:teuthology.orchestra.run.smithi111.stdout:checking bstring.h usability... no 2024-05-27T00:41:18.339 INFO:teuthology.orchestra.run.smithi111.stdout:checking bstring.h presence... no 2024-05-27T00:41:18.339 INFO:teuthology.orchestra.run.smithi111.stdout:checking for bstring.h... no 2024-05-27T00:41:18.364 INFO:teuthology.orchestra.run.smithi111.stdout:checking libgen.h usability... yes 2024-05-27T00:41:18.372 INFO:teuthology.orchestra.run.smithi111.stdout:checking libgen.h presence... yes 2024-05-27T00:41:18.372 INFO:teuthology.orchestra.run.smithi111.stdout:checking for libgen.h... yes 2024-05-27T00:41:18.401 INFO:teuthology.orchestra.run.smithi111.stdout:checking dirent.h usability... yes 2024-05-27T00:41:18.410 INFO:teuthology.orchestra.run.smithi111.stdout:checking dirent.h presence... yes 2024-05-27T00:41:18.410 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dirent.h... yes 2024-05-27T00:41:18.452 INFO:teuthology.orchestra.run.smithi111.stdout:checking errno.h usability... yes 2024-05-27T00:41:18.460 INFO:teuthology.orchestra.run.smithi111.stdout:checking errno.h presence... yes 2024-05-27T00:41:18.460 INFO:teuthology.orchestra.run.smithi111.stdout:checking for errno.h... yes 2024-05-27T00:41:18.488 INFO:teuthology.orchestra.run.smithi111.stdout:checking malloc.h usability... yes 2024-05-27T00:41:18.498 INFO:teuthology.orchestra.run.smithi111.stdout:checking malloc.h presence... yes 2024-05-27T00:41:18.498 INFO:teuthology.orchestra.run.smithi111.stdout:checking for malloc.h... yes 2024-05-27T00:41:18.525 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid.h usability... no 2024-05-27T00:41:18.536 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid.h presence... no 2024-05-27T00:41:18.536 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid.h... no 2024-05-27T00:41:18.563 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid/uuid.h usability... yes 2024-05-27T00:41:18.573 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid/uuid.h presence... yes 2024-05-27T00:41:18.573 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid/uuid.h... yes 2024-05-27T00:41:18.600 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/uuid.h usability... no 2024-05-27T00:41:18.610 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/uuid.h presence... no 2024-05-27T00:41:18.611 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/uuid.h... no 2024-05-27T00:41:18.641 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/file.h usability... yes 2024-05-27T00:41:18.650 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/file.h presence... yes 2024-05-27T00:41:18.650 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/file.h... yes 2024-05-27T00:41:18.677 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fcntl.h usability... yes 2024-05-27T00:41:18.686 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fcntl.h presence... yes 2024-05-27T00:41:18.686 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/fcntl.h... yes 2024-05-27T00:41:18.713 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/syssgi.h usability... no 2024-05-27T00:41:18.725 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/syssgi.h presence... no 2024-05-27T00:41:18.725 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/syssgi.h... no 2024-05-27T00:41:18.765 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/param.h usability... yes 2024-05-27T00:41:18.777 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/param.h presence... yes 2024-05-27T00:41:18.777 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/param.h... yes 2024-05-27T00:41:18.781 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/stat.h... (cached) yes 2024-05-27T00:41:18.809 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/statvfs.h usability... yes 2024-05-27T00:41:18.818 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/statvfs.h presence... yes 2024-05-27T00:41:18.818 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/statvfs.h... yes 2024-05-27T00:41:18.846 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/time.h usability... yes 2024-05-27T00:41:18.855 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/time.h presence... yes 2024-05-27T00:41:18.855 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/time.h... yes 2024-05-27T00:41:18.887 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/ioctl.h usability... yes 2024-05-27T00:41:18.896 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/ioctl.h presence... yes 2024-05-27T00:41:18.896 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/ioctl.h... yes 2024-05-27T00:41:18.929 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/wait.h usability... yes 2024-05-27T00:41:18.939 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/wait.h presence... yes 2024-05-27T00:41:18.940 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/wait.h... yes 2024-05-27T00:41:18.944 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/types.h... (cached) yes 2024-05-27T00:41:18.949 INFO:teuthology.orchestra.run.smithi111.stdout:checking for strings.h... (cached) yes 2024-05-27T00:41:18.977 INFO:teuthology.orchestra.run.smithi111.stdout:checking err.h usability... yes 2024-05-27T00:41:18.985 INFO:teuthology.orchestra.run.smithi111.stdout:checking err.h presence... yes 2024-05-27T00:41:18.985 INFO:teuthology.orchestra.run.smithi111.stdout:checking for err.h... yes 2024-05-27T00:41:19.012 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/falloc.h usability... yes 2024-05-27T00:41:19.020 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/falloc.h presence... yes 2024-05-27T00:41:19.020 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/falloc.h... yes 2024-05-27T00:41:19.047 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-05-27T00:41:19.058 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-05-27T00:41:19.058 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-27T00:41:19.083 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_itable.h usability... no 2024-05-27T00:41:19.093 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_itable.h presence... no 2024-05-27T00:41:19.094 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-27T00:41:19.118 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/platform_defs.h usability... no 2024-05-27T00:41:19.129 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/platform_defs.h presence... no 2024-05-27T00:41:19.129 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/platform_defs.h... no 2024-05-27T00:41:19.153 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfs/ioctl.h usability... no 2024-05-27T00:41:19.164 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfs/ioctl.h presence... no 2024-05-27T00:41:19.164 INFO:teuthology.orchestra.run.smithi111.stdout:checking for btrfs/ioctl.h... no 2024-05-27T00:41:19.189 INFO:teuthology.orchestra.run.smithi111.stdout:checking cifs/ioctl.h usability... no 2024-05-27T00:41:19.199 INFO:teuthology.orchestra.run.smithi111.stdout:checking cifs/ioctl.h presence... no 2024-05-27T00:41:19.200 INFO:teuthology.orchestra.run.smithi111.stdout:checking for cifs/ioctl.h... no 2024-05-27T00:41:19.236 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/mman.h usability... yes 2024-05-27T00:41:19.245 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/mman.h presence... yes 2024-05-27T00:41:19.245 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/mman.h... yes 2024-05-27T00:41:19.272 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/ext4.h usability... no 2024-05-27T00:41:19.283 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/ext4.h presence... no 2024-05-27T00:41:19.283 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/ext4.h... no 2024-05-27T00:41:19.295 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/xfs_log_format.h... no 2024-05-27T00:41:19.342 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid_compare... no 2024-05-27T00:41:19.373 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid_compare in -luuid... yes 2024-05-27T00:41:19.431 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/xfs.h... yes 2024-05-27T00:41:19.444 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/libxfs.h... no 2024-05-27T00:41:19.459 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-27T00:41:19.489 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/xqm.h... yes 2024-05-27T00:41:19.539 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-27T00:41:19.563 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/handle.h usability... yes 2024-05-27T00:41:19.570 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/handle.h presence... yes 2024-05-27T00:41:19.571 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/handle.h... yes 2024-05-27T00:41:19.609 INFO:teuthology.orchestra.run.smithi111.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-27T00:41:19.640 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/acl.h usability... yes 2024-05-27T00:41:19.650 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/acl.h presence... yes 2024-05-27T00:41:19.650 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/acl.h... yes 2024-05-27T00:41:19.690 INFO:teuthology.orchestra.run.smithi111.stdout:checking acl/libacl.h usability... yes 2024-05-27T00:41:19.700 INFO:teuthology.orchestra.run.smithi111.stdout:checking acl/libacl.h presence... yes 2024-05-27T00:41:19.700 INFO:teuthology.orchestra.run.smithi111.stdout:checking for acl/libacl.h... yes 2024-05-27T00:41:19.732 INFO:teuthology.orchestra.run.smithi111.stdout:checking for acl_init in -lacl... yes 2024-05-27T00:41:19.755 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm-ndbm.h usability... no 2024-05-27T00:41:19.766 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm-ndbm.h presence... no 2024-05-27T00:41:19.767 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gdbm-ndbm.h... no 2024-05-27T00:41:19.789 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm/ndbm.h usability... yes 2024-05-27T00:41:19.800 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm/ndbm.h presence... yes 2024-05-27T00:41:19.800 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gdbm/ndbm.h... yes 2024-05-27T00:41:19.835 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_open in -lgdbm... no 2024-05-27T00:41:19.869 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_fetch in -lgdbm... no 2024-05-27T00:41:19.904 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_store in -lgdbm... no 2024-05-27T00:41:19.938 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_close in -lgdbm... no 2024-05-27T00:41:19.969 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-05-27T00:41:20.000 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-05-27T00:41:20.032 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-05-27T00:41:20.066 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-05-27T00:41:20.089 INFO:teuthology.orchestra.run.smithi111.stdout:checking libaio.h usability... yes 2024-05-27T00:41:20.101 INFO:teuthology.orchestra.run.smithi111.stdout:checking libaio.h presence... yes 2024-05-27T00:41:20.101 INFO:teuthology.orchestra.run.smithi111.stdout:checking for libaio.h... yes 2024-05-27T00:41:20.125 INFO:teuthology.orchestra.run.smithi111.stdout:checking liburing.h usability... no 2024-05-27T00:41:20.136 INFO:teuthology.orchestra.run.smithi111.stdout:checking liburing.h presence... no 2024-05-27T00:41:20.136 INFO:teuthology.orchestra.run.smithi111.stdout:checking for liburing.h... no 2024-05-27T00:41:20.163 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/capability.h usability... yes 2024-05-27T00:41:20.173 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/capability.h presence... yes 2024-05-27T00:41:20.173 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/capability.h... yes 2024-05-27T00:41:20.197 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fiemap.h usability... yes 2024-05-27T00:41:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fiemap.h presence... yes 2024-05-27T00:41:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/fiemap.h... yes 2024-05-27T00:41:20.239 INFO:teuthology.orchestra.run.smithi111.stdout:checking for fallocate... yes 2024-05-27T00:41:20.273 INFO:teuthology.orchestra.run.smithi111.stdout:checking for open_by_handle_at... yes 2024-05-27T00:41:20.297 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fs.h usability... yes 2024-05-27T00:41:20.305 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fs.h presence... yes 2024-05-27T00:41:20.305 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/fs.h... yes 2024-05-27T00:41:20.327 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfsutil.h usability... no 2024-05-27T00:41:20.338 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfsutil.h presence... no 2024-05-27T00:41:20.338 INFO:teuthology.orchestra.run.smithi111.stdout:checking for btrfsutil.h... no 2024-05-27T00:41:20.375 INFO:teuthology.orchestra.run.smithi111.stdout:checking for copy_file_range... yes 2024-05-27T00:41:20.411 INFO:teuthology.orchestra.run.smithi111.stdout:checking for SEEK_DATA... yes 2024-05-27T00:41:20.461 INFO:teuthology.orchestra.run.smithi111.stdout:checking for BMV_OF_SHARED... yes 2024-05-27T00:41:20.497 INFO:teuthology.orchestra.run.smithi111.stdout:checking for nftw... yes 2024-05-27T00:41:20.538 INFO:teuthology.orchestra.run.smithi111.stdout:checking for RLIMIT_NOFILE... yes 2024-05-27T00:41:20.570 INFO:teuthology.orchestra.run.smithi111.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-05-27T00:41:20.603 INFO:teuthology.orchestra.run.smithi111.stdout:checking for FICLONE... yes 2024-05-27T00:41:20.637 INFO:teuthology.orchestra.run.smithi111.stdout:checking for renameat2... yes 2024-05-27T00:41:20.670 INFO:teuthology.orchestra.run.smithi111.stdout:checking for reallocarray... yes 2024-05-27T00:41:20.700 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct mount_attr... yes 2024-05-27T00:41:20.734 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_qgroup_limit... yes 2024-05-27T00:41:20.766 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-05-27T00:41:20.799 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-05-27T00:41:20.832 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-05-27T00:41:20.865 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-05-27T00:41:20.897 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-05-27T00:41:20.930 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-05-27T00:41:20.957 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs.h usability... yes 2024-05-27T00:41:20.965 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs.h presence... yes 2024-05-27T00:41:20.965 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/btrfs.h... yes 2024-05-27T00:41:20.995 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs_tree.h usability... yes 2024-05-27T00:41:21.004 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs_tree.h presence... yes 2024-05-27T00:41:21.004 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/btrfs_tree.h... yes 2024-05-27T00:41:21.025 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-05-27T00:41:21.047 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-05-27T00:41:21.063 INFO:teuthology.orchestra.run.smithi111.stdout:configure: creating ./config.status 2024-05-27T00:41:21.356 INFO:teuthology.orchestra.run.smithi111.stdout:config.status: creating include/builddefs 2024-05-27T00:41:21.367 INFO:teuthology.orchestra.run.smithi111.stdout:config.status: creating include/config.h 2024-05-27T00:41:21.375 INFO:teuthology.orchestra.run.smithi111.stdout:config.status: executing libtool commands 2024-05-27T00:41:21.413 INFO:teuthology.orchestra.run.smithi111.stdout:Building include 2024-05-27T00:41:21.416 INFO:teuthology.orchestra.run.smithi111.stdout:Building lib 2024-05-27T00:41:21.420 INFO:teuthology.orchestra.run.smithi111.stdout: [LTDEP] 2024-05-27T00:41:21.562 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dataascii.lo 2024-05-27T00:41:21.671 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] databin.lo 2024-05-27T00:41:21.805 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] datapid.lo 2024-05-27T00:41:21.900 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] file_lock.lo 2024-05-27T00:41:22.033 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] forker.lo 2024-05-27T00:41:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] pattern.lo 2024-05-27T00:41:22.264 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] open_flags.lo 2024-05-27T00:41:22.391 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] random_range.lo 2024-05-27T00:41:22.596 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] string_to_tokens.lo 2024-05-27T00:41:22.690 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] str_to_bytes.lo 2024-05-27T00:41:22.817 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] tlibio.lo 2024-05-27T00:41:23.035 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] write_log.lo 2024-05-27T00:41:23.251 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] random.lo 2024-05-27T00:41:23.351 INFO:teuthology.orchestra.run.smithi111.stdout: [LD] 2024-05-27T00:41:23.433 INFO:teuthology.orchestra.run.smithi111.stdout:Building ltp 2024-05-27T00:41:23.435 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2024-05-27T00:41:23.564 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] doio 2024-05-27T00:41:24.166 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsstress 2024-05-27T00:41:25.479 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsx 2024-05-27T00:41:26.150 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] growfiles 2024-05-27T00:41:26.616 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] iogen 2024-05-27T00:41:26.952 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-stress 2024-05-27T00:41:27.336 INFO:teuthology.orchestra.run.smithi111.stdout:Building src 2024-05-27T00:41:27.342 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2024-05-27T00:41:29.036 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2024-05-27T00:41:29.067 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2024-05-27T00:41:29.074 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2024-05-27T00:41:29.287 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2024-05-27T00:41:29.308 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dirstress 2024-05-27T00:41:29.514 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fill 2024-05-27T00:41:29.647 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fill2 2024-05-27T00:41:29.848 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] getpagesize 2024-05-27T00:41:29.962 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] holes 2024-05-27T00:41:30.129 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] lstat64 2024-05-27T00:41:30.285 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] nametest 2024-05-27T00:41:30.495 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] permname 2024-05-27T00:41:30.669 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] randholes 2024-05-27T00:41:30.899 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] runas 2024-05-27T00:41:31.050 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] truncfile 2024-05-27T00:41:31.204 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] usemem 2024-05-27T00:41:31.331 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mmapcat 2024-05-27T00:41:31.456 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] append_reader 2024-05-27T00:41:31.584 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] append_writer 2024-05-27T00:41:31.712 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dirperf 2024-05-27T00:41:31.908 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] metaperf 2024-05-27T00:41:32.149 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] devzero 2024-05-27T00:41:32.298 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] feature 2024-05-27T00:41:32.483 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] alloc 2024-05-27T00:41:32.662 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fault 2024-05-27T00:41:32.799 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fstest 2024-05-27T00:41:33.006 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_access_root 2024-05-27T00:41:33.136 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] godown 2024-05-27T00:41:33.282 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] resvtest 2024-05-27T00:41:33.439 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] writemod 2024-05-27T00:41:33.562 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] writev_on_pagefault 2024-05-27T00:41:33.699 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] makeextents 2024-05-27T00:41:33.851 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] itrash 2024-05-27T00:41:33.981 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] rename 2024-05-27T00:41:34.099 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] multi_open_unlink 2024-05-27T00:41:34.239 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] unwritten_sync 2024-05-27T00:41:34.393 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] genhashnames 2024-05-27T00:41:34.558 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_holes 2024-05-27T00:41:34.683 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_writev 2024-05-27T00:41:34.809 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_truncate_cmtime 2024-05-27T00:41:34.946 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dirhash_collide 2024-05-27T00:41:35.111 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_rename_overwrite 2024-05-27T00:41:35.230 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] holetest 2024-05-27T00:41:35.426 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_truncate_self 2024-05-27T00:41:35.545 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] af_unix 2024-05-27T00:41:35.679 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_stale_pmd 2024-05-27T00:41:35.808 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_cow_race 2024-05-27T00:41:35.949 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_fallocate 2024-05-27T00:41:36.077 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsync-err 2024-05-27T00:41:36.238 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_write_ro 2024-05-27T00:41:36.363 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-27T00:41:36.499 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-27T00:41:36.631 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_ofd_locks 2024-05-27T00:41:36.800 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_collision 2024-05-27T00:41:36.979 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mmap-write-concurrent 2024-05-27T00:41:37.124 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_get_file_time 2024-05-27T00:41:37.248 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_create_short_dirs 2024-05-27T00:41:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_create_long_dirs 2024-05-27T00:41:37.536 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_enospc 2024-05-27T00:41:37.710 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_writev_overlap 2024-05-27T00:41:37.854 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] checkpoint_journal 2024-05-27T00:41:37.985 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mmap-rw-fault 2024-05-27T00:41:38.143 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] allocstale 2024-05-27T00:41:38.286 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_cow_memory_failure 2024-05-27T00:41:38.432 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fake-dump-rootino 2024-05-27T00:41:38.560 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dio-buf-fault 2024-05-27T00:41:38.690 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] rewinddir-test 2024-05-27T00:41:38.836 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] readdir-while-renames 2024-05-27T00:41:38.976 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] xfsctl 2024-05-27T00:41:39.138 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bstat 2024-05-27T00:41:39.317 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mtab 2024-05-27T00:41:39.479 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] getdevicesize 2024-05-27T00:41:39.599 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] preallo_rw_pattern_reader 2024-05-27T00:41:39.735 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] preallo_rw_pattern_writer 2024-05-27T00:41:39.869 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] ftrunc 2024-05-27T00:41:39.995 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] trunc 2024-05-27T00:41:40.134 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fs_perms 2024-05-27T00:41:40.272 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] testx 2024-05-27T00:41:40.383 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] looptest 2024-05-27T00:41:40.549 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] locktest 2024-05-27T00:41:40.820 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] unwritten_mmap 2024-05-27T00:41:40.960 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bulkstat_unlink_test 2024-05-27T00:41:41.132 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] deduperace 2024-05-27T00:41:41.299 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bulkstat_unlink_test_modified 2024-05-27T00:41:41.467 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_dir_offset 2024-05-27T00:41:41.594 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_futimens 2024-05-27T00:41:41.713 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_immutable 2024-05-27T00:41:42.271 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] stale_handle 2024-05-27T00:41:42.417 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] pwrite_mmap_blocked 2024-05-27T00:41:42.547 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_dir_offset2 2024-05-27T00:41:42.713 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] seek_sanity_test 2024-05-27T00:41:43.110 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] seek_copy_test 2024-05-27T00:41:43.262 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_readdir_1 2024-05-27T00:41:43.388 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_readdir_2 2024-05-27T00:41:43.519 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsync-tester 2024-05-27T00:41:43.738 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] nsexec 2024-05-27T00:41:43.891 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] cloner 2024-05-27T00:41:44.044 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] renameat2 2024-05-27T00:41:44.185 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_getcwd 2024-05-27T00:41:44.321 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] e4compact 2024-05-27T00:41:44.495 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] test-nextquota 2024-05-27T00:41:44.642 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] punch-alternating 2024-05-27T00:41:44.788 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-27T00:41:44.934 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] listxattr 2024-05-27T00:41:45.059 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dio-interleaved 2024-05-27T00:41:45.196 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_dir_type 2024-05-27T00:41:45.330 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dio-invalidate-cache 2024-05-27T00:41:45.506 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] stat_test 2024-05-27T00:41:45.767 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_encrypted_d_revalidate 2024-05-27T00:41:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] attr_replace_test 2024-05-27T00:41:46.046 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] swapon 2024-05-27T00:41:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mkswap 2024-05-27T00:41:46.317 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_attr_corruption 2024-05-27T00:41:46.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:41:46.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:41:46.445 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_open_tmpfiles 2024-05-27T00:41:46.599 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fscrypt-crypt-util 2024-05-27T00:41:47.195 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bulkstat_null_ocount 2024-05-27T00:41:47.331 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] splice-test 2024-05-27T00:41:47.487 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] chprojid_fail 2024-05-27T00:41:47.625 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] detached_mounts_propagation 2024-05-27T00:41:47.785 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] ext4_resize 2024-05-27T00:41:47.913 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_readdir_3 2024-05-27T00:41:48.082 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] splice2pipe 2024-05-27T00:41:48.220 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] uuid_ioctl 2024-05-27T00:41:48.353 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_snapshot_deleted_subvolume 2024-05-27T00:41:48.494 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fiemap-fault 2024-05-27T00:41:48.629 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fiemap-tester 2024-05-27T00:41:48.860 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_stripealign 2024-05-27T00:41:48.992 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] open_by_handle 2024-05-27T00:41:49.210 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dbtest 2024-05-27T00:41:49.428 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_dio 2024-05-27T00:41:49.567 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] xfsfind 2024-05-27T00:41:49.734 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_reflink_read_race 2024-05-27T00:41:49.902 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fssum 2024-05-27T00:41:50.299 INFO:teuthology.orchestra.run.smithi111.stdout:Building log-writes 2024-05-27T00:41:50.304 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] replay-log 2024-05-27T00:41:50.586 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2024-05-27T00:41:50.589 INFO:teuthology.orchestra.run.smithi111.stdout:Building aio-dio-regress 2024-05-27T00:41:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aiocp 2024-05-27T00:41:50.793 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-27T00:41:50.939 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-append-write-read-race 2024-05-27T00:41:51.089 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-cow-race 2024-05-27T00:41:51.219 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-cycle-write 2024-05-27T00:41:51.377 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-eof-race 2024-05-27T00:41:51.535 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-extend-stat 2024-05-27T00:41:51.676 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-fcntl-race 2024-05-27T00:41:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-hole-filling-race 2024-05-27T00:41:51.943 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-invalidate-failure 2024-05-27T00:41:52.080 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-invalidate-readahead 2024-05-27T00:41:52.222 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aiodio_sparse2 2024-05-27T00:41:52.411 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-subblock-eof-read 2024-05-27T00:41:52.532 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-write-verify 2024-05-27T00:41:52.704 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-27T00:41:52.819 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-27T00:41:52.930 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-last-ref-held-by-io 2024-05-27T00:41:53.079 INFO:teuthology.orchestra.run.smithi111.stdout:Building vfs 2024-05-27T00:41:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] vfstest 2024-05-27T00:41:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mount-idmapped 2024-05-27T00:41:56.717 INFO:teuthology.orchestra.run.smithi111.stdout:Building m4 2024-05-27T00:41:56.720 INFO:teuthology.orchestra.run.smithi111.stdout:Building common 2024-05-27T00:41:56.723 INFO:teuthology.orchestra.run.smithi111.stdout:Building tests 2024-05-27T00:41:56.729 INFO:teuthology.orchestra.run.smithi111.stdout:Building btrfs 2024-05-27T00:41:56.735 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/btrfs/group.list 2024-05-27T00:41:56.798 INFO:teuthology.orchestra.run.smithi111.stdout:Building ceph 2024-05-27T00:41:56.803 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ceph/group.list 2024-05-27T00:41:56.818 INFO:teuthology.orchestra.run.smithi111.stdout:Building cifs 2024-05-27T00:41:56.823 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/cifs/group.list 2024-05-27T00:41:56.835 INFO:teuthology.orchestra.run.smithi111.stdout:Building ext4 2024-05-27T00:41:56.841 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ext4/group.list 2024-05-27T00:41:56.880 INFO:teuthology.orchestra.run.smithi111.stdout:Building f2fs 2024-05-27T00:41:56.885 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/f2fs/group.list 2024-05-27T00:41:56.903 INFO:teuthology.orchestra.run.smithi111.stdout:Building generic 2024-05-27T00:41:56.909 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/generic/group.list 2024-05-27T00:41:57.010 INFO:teuthology.orchestra.run.smithi111.stdout:Building nfs 2024-05-27T00:41:57.015 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/nfs/group.list 2024-05-27T00:41:57.030 INFO:teuthology.orchestra.run.smithi111.stdout:Building ocfs2 2024-05-27T00:41:57.035 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ocfs2/group.list 2024-05-27T00:41:57.048 INFO:teuthology.orchestra.run.smithi111.stdout:Building overlay 2024-05-27T00:41:57.054 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/overlay/group.list 2024-05-27T00:41:57.095 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2024-05-27T00:41:57.100 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/perf/group.list 2024-05-27T00:41:57.112 INFO:teuthology.orchestra.run.smithi111.stdout:Building selftest 2024-05-27T00:41:57.117 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/selftest/group.list 2024-05-27T00:41:57.130 INFO:teuthology.orchestra.run.smithi111.stdout:Building tmpfs 2024-05-27T00:41:57.135 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/tmpfs/group.list 2024-05-27T00:41:57.148 INFO:teuthology.orchestra.run.smithi111.stdout:Building udf 2024-05-27T00:41:57.153 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/udf/group.list 2024-05-27T00:41:57.165 INFO:teuthology.orchestra.run.smithi111.stdout:Building xfs 2024-05-27T00:41:57.172 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/xfs/group.list 2024-05-27T00:41:57.273 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo make install) 2024-05-27T00:41:57.308 INFO:teuthology.orchestra.run.smithi111.stdout:Building include 2024-05-27T00:41:57.312 INFO:teuthology.orchestra.run.smithi111.stdout:Building lib 2024-05-27T00:41:57.315 INFO:teuthology.orchestra.run.smithi111.stdout:Building ltp 2024-05-27T00:41:57.319 INFO:teuthology.orchestra.run.smithi111.stdout:Building src 2024-05-27T00:41:57.354 INFO:teuthology.orchestra.run.smithi111.stdout:Building log-writes 2024-05-27T00:41:57.357 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2024-05-27T00:41:57.360 INFO:teuthology.orchestra.run.smithi111.stdout:Building aio-dio-regress 2024-05-27T00:41:57.364 INFO:teuthology.orchestra.run.smithi111.stdout:Building vfs 2024-05-27T00:41:57.368 INFO:teuthology.orchestra.run.smithi111.stdout:Building m4 2024-05-27T00:41:57.372 INFO:teuthology.orchestra.run.smithi111.stdout:Building common 2024-05-27T00:41:57.375 INFO:teuthology.orchestra.run.smithi111.stdout:Building tests 2024-05-27T00:41:57.380 INFO:teuthology.orchestra.run.smithi111.stdout:Building btrfs 2024-05-27T00:41:57.386 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/btrfs/group.list 2024-05-27T00:41:57.448 INFO:teuthology.orchestra.run.smithi111.stdout:Building ceph 2024-05-27T00:41:57.453 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ceph/group.list 2024-05-27T00:41:57.468 INFO:teuthology.orchestra.run.smithi111.stdout:Building cifs 2024-05-27T00:41:57.473 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/cifs/group.list 2024-05-27T00:41:57.486 INFO:teuthology.orchestra.run.smithi111.stdout:Building ext4 2024-05-27T00:41:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ext4/group.list 2024-05-27T00:41:57.530 INFO:teuthology.orchestra.run.smithi111.stdout:Building f2fs 2024-05-27T00:41:57.535 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/f2fs/group.list 2024-05-27T00:41:57.552 INFO:teuthology.orchestra.run.smithi111.stdout:Building generic 2024-05-27T00:41:57.559 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/generic/group.list 2024-05-27T00:41:57.659 INFO:teuthology.orchestra.run.smithi111.stdout:Building nfs 2024-05-27T00:41:57.664 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/nfs/group.list 2024-05-27T00:41:57.679 INFO:teuthology.orchestra.run.smithi111.stdout:Building ocfs2 2024-05-27T00:41:57.684 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ocfs2/group.list 2024-05-27T00:41:57.698 INFO:teuthology.orchestra.run.smithi111.stdout:Building overlay 2024-05-27T00:41:57.703 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/overlay/group.list 2024-05-27T00:41:57.745 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2024-05-27T00:41:57.750 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/perf/group.list 2024-05-27T00:41:57.761 INFO:teuthology.orchestra.run.smithi111.stdout:Building selftest 2024-05-27T00:41:57.766 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/selftest/group.list 2024-05-27T00:41:57.779 INFO:teuthology.orchestra.run.smithi111.stdout:Building tmpfs 2024-05-27T00:41:57.784 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/tmpfs/group.list 2024-05-27T00:41:57.797 INFO:teuthology.orchestra.run.smithi111.stdout:Building udf 2024-05-27T00:41:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/udf/group.list 2024-05-27T00:41:57.814 INFO:teuthology.orchestra.run.smithi111.stdout:Building xfs 2024-05-27T00:41:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/xfs/group.list 2024-05-27T00:41:57.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-05-27T00:41:57.921 INFO:teuthology.orchestra.run.smithi111.stdout:gmake[1]: Nothing to be done for 'install'. 2024-05-27T00:41:57.922 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-05-27T00:41:57.924 INFO:teuthology.orchestra.run.smithi111.stdout:gmake[1]: Nothing to be done for 'install'. 2024-05-27T00:41:57.924 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-05-27T00:41:57.926 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-05-27T00:41:57.935 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-05-27T00:41:58.004 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-05-27T00:41:58.012 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-05-27T00:41:58.029 INFO:teuthology.orchestra.run.smithi111.stdout:Building log-writes 2024-05-27T00:41:58.033 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2024-05-27T00:41:58.036 INFO:teuthology.orchestra.run.smithi111.stdout:Building aio-dio-regress 2024-05-27T00:41:58.040 INFO:teuthology.orchestra.run.smithi111.stdout:Building vfs 2024-05-27T00:41:58.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C log-writes install 2024-05-27T00:41:58.044 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-05-27T00:41:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-05-27T00:41:58.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C perf install 2024-05-27T00:41:58.063 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-05-27T00:41:58.072 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-05-27T00:41:58.093 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-05-27T00:41:58.095 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-05-27T00:41:58.104 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-05-27T00:41:58.154 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C vfs install 2024-05-27T00:41:58.155 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-05-27T00:41:58.164 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-05-27T00:41:58.176 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-05-27T00:41:58.184 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-05-27T00:41:59.373 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-05-27T00:41:59.468 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-05-27T00:41:59.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-05-27T00:41:59.494 INFO:teuthology.orchestra.run.smithi111.stdout:gmake[1]: Nothing to be done for 'install'. 2024-05-27T00:41:59.494 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-05-27T00:41:59.496 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-05-27T00:41:59.504 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-05-27T00:41:59.638 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-05-27T00:41:59.639 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/btrfs/ install 2024-05-27T00:41:59.642 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/btrfs/group.list 2024-05-27T00:41:59.706 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-05-27T00:41:59.715 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 313 314 315 316 317 318 320 330 /var/lib/xfstests/tests/btrfs 2024-05-27T00:42:00.544 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-05-27T00:42:00.553 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 313.out 314.out 315.out 316.out 317.out 318.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-05-27T00:42:01.388 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ceph/ install 2024-05-27T00:42:01.390 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ceph/group.list 2024-05-27T00:42:01.404 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-05-27T00:42:01.413 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-05-27T00:42:01.432 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-05-27T00:42:01.441 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-05-27T00:42:01.460 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/cifs/ install 2024-05-27T00:42:01.462 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/cifs/group.list 2024-05-27T00:42:01.474 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-05-27T00:42:01.483 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-05-27T00:42:01.491 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-05-27T00:42:01.500 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-05-27T00:42:01.508 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ext4/ install 2024-05-27T00:42:01.511 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ext4/group.list 2024-05-27T00:42:01.550 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-05-27T00:42:01.558 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-05-27T00:42:01.739 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-05-27T00:42:01.748 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-05-27T00:42:01.937 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/f2fs/ install 2024-05-27T00:42:01.940 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/f2fs/group.list 2024-05-27T00:42:01.956 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-05-27T00:42:01.965 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-05-27T00:42:01.976 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-05-27T00:42:01.985 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-05-27T00:42:01.996 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/generic/ install 2024-05-27T00:42:01.999 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/generic/group.list 2024-05-27T00:42:02.103 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-05-27T00:42:02.111 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 /var/lib/xfstests/tests/generic 2024-05-27T00:42:04.055 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-05-27T00:42:04.064 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out /var/lib/xfstests/tests/generic 2024-05-27T00:42:06.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/nfs/ install 2024-05-27T00:42:06.039 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/nfs/group.list 2024-05-27T00:42:06.052 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-05-27T00:42:06.061 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-05-27T00:42:06.069 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-05-27T00:42:06.078 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-05-27T00:42:06.087 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ocfs2/ install 2024-05-27T00:42:06.089 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/ocfs2/group.list 2024-05-27T00:42:06.102 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-05-27T00:42:06.110 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-05-27T00:42:06.119 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-05-27T00:42:06.127 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-05-27T00:42:06.137 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/overlay/ install 2024-05-27T00:42:06.139 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/overlay/group.list 2024-05-27T00:42:06.181 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-05-27T00:42:06.189 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-05-27T00:42:06.464 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-05-27T00:42:06.473 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-05-27T00:42:06.748 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/perf/ install 2024-05-27T00:42:06.750 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/perf/group.list 2024-05-27T00:42:06.761 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-05-27T00:42:06.770 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-05-27T00:42:06.779 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-05-27T00:42:06.787 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-05-27T00:42:06.796 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/selftest/ install 2024-05-27T00:42:06.799 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/selftest/group.list 2024-05-27T00:42:06.811 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-05-27T00:42:06.820 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-05-27T00:42:06.841 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-05-27T00:42:06.850 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-05-27T00:42:06.872 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/tmpfs/ install 2024-05-27T00:42:06.874 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/tmpfs/group.list 2024-05-27T00:42:06.887 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-05-27T00:42:06.895 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-05-27T00:42:06.904 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-05-27T00:42:06.913 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-05-27T00:42:06.922 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/udf/ install 2024-05-27T00:42:06.924 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/udf/group.list 2024-05-27T00:42:06.935 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-05-27T00:42:06.943 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-05-27T00:42:06.952 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-05-27T00:42:06.960 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-05-27T00:42:06.969 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/xfs/ install 2024-05-27T00:42:06.972 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.pEH4Qq3ph1xfstests-dev/tests/xfs/group.list 2024-05-27T00:42:07.073 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-05-27T00:42:07.081 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 612 613 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-05-27T00:42:08.905 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-05-27T00:42:08.914 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 612.out 613.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-05-27T00:42:10.761 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-05-27T00:42:10.769 INFO:teuthology.orchestra.run.smithi111.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-05-27T00:42:10.778 INFO:teuthology.orchestra.run.smithi111.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-05-27T00:42:10.786 INFO:teuthology.orchestra.run.smithi111.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-05-27T00:42:10.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T00:42:10.797 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec ls tests/generic) 2024-05-27T00:42:10.854 INFO:teuthology.orchestra.run.smithi111.stdout:001 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:001.out 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:002 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:002.out 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:003 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:003.out 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:004 2024-05-27T00:42:10.874 INFO:teuthology.orchestra.run.smithi111.stdout:004.out 2024-05-27T00:42:10.875 INFO:teuthology.orchestra.run.smithi111.stdout:005 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:005.out 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:006 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:006.out 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:007 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:007.out 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:008 2024-05-27T00:42:10.877 INFO:teuthology.orchestra.run.smithi111.stdout:008.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:009 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:009.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:010 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:010.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:011 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:011.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:012 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:012.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:013 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:013.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:014 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:014.out 2024-05-27T00:42:10.878 INFO:teuthology.orchestra.run.smithi111.stdout:015 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:015.out 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:016 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:016.out 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:017 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:017.out 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:018 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:018.out 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:019 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:019.out 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:020 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:020.out 2024-05-27T00:42:10.879 INFO:teuthology.orchestra.run.smithi111.stdout:021 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:021.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:022 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:022.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:023 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:023.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:024 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:024.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:025 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:025.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:026 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:026.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:027 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:027.out 2024-05-27T00:42:10.880 INFO:teuthology.orchestra.run.smithi111.stdout:028 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:028.out 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:029 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:029.out 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:030 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:030.out 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:031 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:031.out 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:032 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:032.out 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:033 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:033.out 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:034 2024-05-27T00:42:10.881 INFO:teuthology.orchestra.run.smithi111.stdout:034.out 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:035 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:035.cfg 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:035.out.default 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:035.out.nfs 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:036 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:036.out 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:037 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:037.out 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:038 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:038.out 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:039 2024-05-27T00:42:10.882 INFO:teuthology.orchestra.run.smithi111.stdout:039.out 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:040 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:040.out 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:041 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:041.out 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:042 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:042.out 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:043 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:043.out 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:044 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:044.out 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:045 2024-05-27T00:42:10.883 INFO:teuthology.orchestra.run.smithi111.stdout:045.out 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:046 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:046.out 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:047 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:047.out 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:048 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:048.out 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:049 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:049.out 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:050 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:050.cfg 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:050.out.default 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:050.out.nojournal 2024-05-27T00:42:10.884 INFO:teuthology.orchestra.run.smithi111.stdout:050.out.xfsquota 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:051 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:051.out 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:052 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:052.out 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:053 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:053.out 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:054 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:054.out 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:055 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:055.out 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:056 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:056.out 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:057 2024-05-27T00:42:10.885 INFO:teuthology.orchestra.run.smithi111.stdout:057.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:058 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:058.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:059 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:059.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:060 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:060.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:061 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:061.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:062 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:062.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:063 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:063.out 2024-05-27T00:42:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:064 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:064.out 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:065 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:065.out 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:066 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:066.out 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:067 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:067.out 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:068 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:068.out 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:069 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:069.out 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:070 2024-05-27T00:42:10.887 INFO:teuthology.orchestra.run.smithi111.stdout:070.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:071 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:071.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:072 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:072.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:073 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:073.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:074 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:074.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:075 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:075.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:076 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:076.out 2024-05-27T00:42:10.888 INFO:teuthology.orchestra.run.smithi111.stdout:077 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:077.out 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:078 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:078.out 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:079 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:079.out 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:080 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:080.out 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:081 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:081.out 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:082 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:082.out 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:083 2024-05-27T00:42:10.889 INFO:teuthology.orchestra.run.smithi111.stdout:083.out 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:084 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:084.out 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:085 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:085.out 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:086 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:086.out 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:087 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:087.out 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:088 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:088.out 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:089 2024-05-27T00:42:10.890 INFO:teuthology.orchestra.run.smithi111.stdout:089.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:090 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:090.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:091 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:091.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:092 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:092.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:093 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:093.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:094 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:094.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:095 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:095.out 2024-05-27T00:42:10.891 INFO:teuthology.orchestra.run.smithi111.stdout:096 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:096.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:097 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:097.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:098 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:098.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:099 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:099.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:100 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:100.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:101 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:101.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:102 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:102.out 2024-05-27T00:42:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:103 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:103.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:104 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:104.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:105 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:105.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:106 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:106.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:107 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:107.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:108 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:108.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:109 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:109.out 2024-05-27T00:42:10.893 INFO:teuthology.orchestra.run.smithi111.stdout:110 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:110.out 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:111 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:111.out 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:112 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:112.out 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:113 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:113.out 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:114 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:114.out 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:115 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:115.out 2024-05-27T00:42:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:116 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:116.out 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:117 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:117.out 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:118 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:118.out 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:119 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:119.out 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:120 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:120.out 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:121 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:121.out 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:122 2024-05-27T00:42:10.895 INFO:teuthology.orchestra.run.smithi111.stdout:122.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:123 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:123.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:124 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:124.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:125 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:125.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:126 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:126.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:127 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:127.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:128 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:128.out 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:129 2024-05-27T00:42:10.896 INFO:teuthology.orchestra.run.smithi111.stdout:129.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:130 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:130.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:131 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:131.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:132 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:132.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:133 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:133.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:134 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:134.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:135 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:135.out 2024-05-27T00:42:10.897 INFO:teuthology.orchestra.run.smithi111.stdout:136 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:136.out 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:137 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:137.out 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:138 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:138.out 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:139 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:139.out 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:140 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:140.out 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:141 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:141.out 2024-05-27T00:42:10.898 INFO:teuthology.orchestra.run.smithi111.stdout:142 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:142.out 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:143 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:143.out 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:144 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:144.out 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:145 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:145.out 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:146 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:146.out 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:147 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:147.out 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:148 2024-05-27T00:42:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:148.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:149 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:149.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:150 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:150.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:151 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:151.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:152 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:152.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:153 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:153.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:154 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:154.out 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:155 2024-05-27T00:42:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:155.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:156 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:156.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:157 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:157.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:158 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:158.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:159 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:159.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:160 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:160.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:161 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:161.out 2024-05-27T00:42:10.901 INFO:teuthology.orchestra.run.smithi111.stdout:162 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:162.out 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:163 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:163.out 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:164 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:164.out 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:165 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:165.out 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:166 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:166.out 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:167 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:167.out 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:168 2024-05-27T00:42:10.902 INFO:teuthology.orchestra.run.smithi111.stdout:168.out 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:169 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:169.out 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:170 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:170.out 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:171 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:171.out 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:172 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:172.out 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:173 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:173.out 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:174 2024-05-27T00:42:10.903 INFO:teuthology.orchestra.run.smithi111.stdout:174.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:175 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:175.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:176 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:176.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:177 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:177.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:178 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:178.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:179 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:179.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:180 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:180.out 2024-05-27T00:42:10.904 INFO:teuthology.orchestra.run.smithi111.stdout:181 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:181.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:182 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:182.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:183 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:183.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:184 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:184.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:185 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:185.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:186 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:186.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:187 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:187.out 2024-05-27T00:42:10.905 INFO:teuthology.orchestra.run.smithi111.stdout:188 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:188.out 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:189 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:189.out 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:190 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:190.out 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:191 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:191.out 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:192 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:192.out 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:193 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:193.out 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:194 2024-05-27T00:42:10.906 INFO:teuthology.orchestra.run.smithi111.stdout:194.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:195 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:195.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:196 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:196.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:197 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:197.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:198 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:198.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:199 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:199.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:200 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:200.out 2024-05-27T00:42:10.907 INFO:teuthology.orchestra.run.smithi111.stdout:201 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:201.out 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:202 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:202.out 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:203 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:203.out 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:204 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:204.out 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:205 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:205.out 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:206 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:206.out 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:207 2024-05-27T00:42:10.908 INFO:teuthology.orchestra.run.smithi111.stdout:207.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:208 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:208.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:209 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:209.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:210 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:210.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:211 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:211.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:212 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:212.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:213 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:213.out 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:214 2024-05-27T00:42:10.909 INFO:teuthology.orchestra.run.smithi111.stdout:214.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:215 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:215.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:216 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:216.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:217 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:217.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:218 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:218.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:219 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:219.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:220 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:220.out 2024-05-27T00:42:10.910 INFO:teuthology.orchestra.run.smithi111.stdout:221 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:221.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:222 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:222.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:223 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:223.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:224 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:224.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:225 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:225.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:226 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:226.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:227 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:227.out 2024-05-27T00:42:10.911 INFO:teuthology.orchestra.run.smithi111.stdout:228 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:228.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:229 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:229.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:230 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:230.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:231 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:231.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:232 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:232.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:233 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:233.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:234 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:234.out 2024-05-27T00:42:10.912 INFO:teuthology.orchestra.run.smithi111.stdout:235 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:235.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:236 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:236.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:237 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:237.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:238 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:238.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:239 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:239.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:240 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:240.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:241 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:241.out 2024-05-27T00:42:10.913 INFO:teuthology.orchestra.run.smithi111.stdout:242 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:242.out 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:243 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:243.out 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:244 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:244.out 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:245 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:245.out 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:246 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:246.out 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:247 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:247.out 2024-05-27T00:42:10.914 INFO:teuthology.orchestra.run.smithi111.stdout:248 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:248.out 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:249 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:249.out 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:250 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:250.out 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:251 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:251.out 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:252 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:252.out 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:253 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:253.out 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:254 2024-05-27T00:42:10.915 INFO:teuthology.orchestra.run.smithi111.stdout:254.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:255 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:255.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:256 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:256.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:257 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:257.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:258 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:258.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:259 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:259.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:260 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:260.out 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:261 2024-05-27T00:42:10.916 INFO:teuthology.orchestra.run.smithi111.stdout:261.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:262 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:262.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:263 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:263.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:264 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:264.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:265 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:265.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:266 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:266.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:267 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:267.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:268 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:268.out 2024-05-27T00:42:10.917 INFO:teuthology.orchestra.run.smithi111.stdout:269 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:269.out 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:270 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:270.out 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:271 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:271.out 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:272 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:272.out 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:273 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:273.out 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:274 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:274.out 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:275 2024-05-27T00:42:10.918 INFO:teuthology.orchestra.run.smithi111.stdout:275.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:276 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:276.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:277 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:277.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:278 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:278.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:279 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:279.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:280 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:280.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:281 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:281.out 2024-05-27T00:42:10.919 INFO:teuthology.orchestra.run.smithi111.stdout:282 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:282.out 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:283 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:283.out 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:284 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:284.out 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:285 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:285.out 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:286 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:286.out 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:287 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:287.out 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:288 2024-05-27T00:42:10.920 INFO:teuthology.orchestra.run.smithi111.stdout:288.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:289 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:289.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:290 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:290.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:291 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:291.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:292 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:292.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:293 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:293.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:294 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:294.out 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:295 2024-05-27T00:42:10.921 INFO:teuthology.orchestra.run.smithi111.stdout:295.out 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:296 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:296.out 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:297 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:297.out 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:298 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:298.out 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:299 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:299.out 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:300 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:300.out 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:301 2024-05-27T00:42:10.922 INFO:teuthology.orchestra.run.smithi111.stdout:301.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:302 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:302.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:303 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:303.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:304 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:304.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:305 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:305.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:306 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:306.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:307 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:307.out 2024-05-27T00:42:10.923 INFO:teuthology.orchestra.run.smithi111.stdout:308 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:308.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:309 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:309.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:310 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:310.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:311 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:311.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:312 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:312.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:313 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:313.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:314 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:314.out 2024-05-27T00:42:10.924 INFO:teuthology.orchestra.run.smithi111.stdout:315 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:315.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:316 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:316.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:317 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:317.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:318 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:318.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:319 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:319.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:320 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:320.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:321 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:321.out 2024-05-27T00:42:10.925 INFO:teuthology.orchestra.run.smithi111.stdout:322 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:322.out 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:323 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:323.out 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:324 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:324.out 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:325 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:325.out 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:326 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:326.out 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:327 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:327.out 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:328 2024-05-27T00:42:10.926 INFO:teuthology.orchestra.run.smithi111.stdout:328.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:329 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:329.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:330 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:330.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:331 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:331.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:332 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:332.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:333 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:333.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:334 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:334.out 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:335 2024-05-27T00:42:10.927 INFO:teuthology.orchestra.run.smithi111.stdout:335.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:336 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:336.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:337 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:337.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:338 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:338.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:339 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:339.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:340 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:340.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:341 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:341.out 2024-05-27T00:42:10.928 INFO:teuthology.orchestra.run.smithi111.stdout:342 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:342.out 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:343 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:343.out 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:344 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:344.out 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:345 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:345.out 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:346 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:346.out 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:347 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:347.out 2024-05-27T00:42:10.929 INFO:teuthology.orchestra.run.smithi111.stdout:348 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:348.out 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:349 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:349.out 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:350 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:350.out 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:351 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:351.out 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:352 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:352.out 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:353 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:353.out 2024-05-27T00:42:10.930 INFO:teuthology.orchestra.run.smithi111.stdout:354 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:354.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:355 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:355.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:356 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:356.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:357 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:357.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:358 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:358.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:359 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:359.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:360 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:360.out 2024-05-27T00:42:10.931 INFO:teuthology.orchestra.run.smithi111.stdout:361 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:361.out 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:362 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:362.out 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:363 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:363.out 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:364 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:364.out 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:365 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:365.out 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:366 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:366.out 2024-05-27T00:42:10.932 INFO:teuthology.orchestra.run.smithi111.stdout:367 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:367.out 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:368 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:368.out 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:369 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:369.out 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:370 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:370.out 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:371 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:371.out 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:372 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:372.out 2024-05-27T00:42:10.933 INFO:teuthology.orchestra.run.smithi111.stdout:373 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:373.out 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:374 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:374.out 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:375 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:375.out 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:376 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:376.out 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:377 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:377.out 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:378 2024-05-27T00:42:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:378.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:379 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:379.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:380 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:380.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:381 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:381.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:382 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:382.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:383 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:383.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:384 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:384.out 2024-05-27T00:42:10.935 INFO:teuthology.orchestra.run.smithi111.stdout:385 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:385.out 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:386 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:386.out 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:387 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:387.out 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:388 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:388.out 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:389 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:389.out 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:390 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:390.out 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:391 2024-05-27T00:42:10.936 INFO:teuthology.orchestra.run.smithi111.stdout:391.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:392 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:392.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:393 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:393.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:394 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:394.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:395 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:395.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:396 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:396.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:397 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:397.out 2024-05-27T00:42:10.937 INFO:teuthology.orchestra.run.smithi111.stdout:398 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:398.out 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:399 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:399.out 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:400 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:400.out 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:401 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:401.out 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:402 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:402.out 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:403 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:403.out 2024-05-27T00:42:10.938 INFO:teuthology.orchestra.run.smithi111.stdout:404 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:404.out 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:405 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:405.out 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:406 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:406.out 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:407 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:407.out 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:408 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:408.out 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:409 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:409.out 2024-05-27T00:42:10.939 INFO:teuthology.orchestra.run.smithi111.stdout:410 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:410.out 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:411 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:411.out 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:412 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:412.out 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:413 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:413.out 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:414 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:414.out 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:415 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:415.out 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:416 2024-05-27T00:42:10.940 INFO:teuthology.orchestra.run.smithi111.stdout:416.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:417 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:417.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:418 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:418.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:419 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:419.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:420 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:420.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:421 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:421.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:422 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:422.out 2024-05-27T00:42:10.941 INFO:teuthology.orchestra.run.smithi111.stdout:423 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:423.out 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:424 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:424.out 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:425 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:425.out 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:426 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:426.out 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:427 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:427.out 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:428 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:428.out 2024-05-27T00:42:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:429 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:429.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:430 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:430.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:431 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:431.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:432 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:432.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:433 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:433.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:434 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:434.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:435 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:435.out 2024-05-27T00:42:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:436 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:436.out 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:437 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:437.out 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:438 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:438.out 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:439 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:439.out 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:440 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:440.out 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:441 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:441.out 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:442 2024-05-27T00:42:10.944 INFO:teuthology.orchestra.run.smithi111.stdout:442.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:443 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:443.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:444 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:444.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:445 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:445.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:446 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:446.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:447 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:447.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:448 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:448.out 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:449 2024-05-27T00:42:10.945 INFO:teuthology.orchestra.run.smithi111.stdout:449.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:450 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:450.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:451 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:451.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:452 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:452.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:453 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:453.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:454 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:454.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:455 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:455.out 2024-05-27T00:42:10.946 INFO:teuthology.orchestra.run.smithi111.stdout:456 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:456.out 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:457 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:457.out 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:458 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:458.out 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:459 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:459.out 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:460 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:460.out 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:461 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:461.out 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:462 2024-05-27T00:42:10.947 INFO:teuthology.orchestra.run.smithi111.stdout:462.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:463 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:463.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:464 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:464.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:465 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:465.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:466 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:466.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:467 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:467.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:468 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:468.out 2024-05-27T00:42:10.948 INFO:teuthology.orchestra.run.smithi111.stdout:469 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:469.out 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:470 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:470.out 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:471 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:471.out 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:472 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:472.out 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:473 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:473.out 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:474 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:474.out 2024-05-27T00:42:10.949 INFO:teuthology.orchestra.run.smithi111.stdout:475 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:475.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:476 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:476.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:477 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:477.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:478 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:478.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:479 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:479.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:480 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:480.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:481 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:481.out 2024-05-27T00:42:10.950 INFO:teuthology.orchestra.run.smithi111.stdout:482 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:482.out 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:483 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:483.out 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:484 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:484.out 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:485 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:485.out 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:486 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:486.out 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:487 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:487.out 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:488 2024-05-27T00:42:10.951 INFO:teuthology.orchestra.run.smithi111.stdout:488.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:489 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:489.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:490 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:490.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:491 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:491.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:492 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:492.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:493 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:493.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:494 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:494.out 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:495 2024-05-27T00:42:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:495.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:496 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:496.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:497 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:497.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:498 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:498.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:499 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:499.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:500 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:500.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:501 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:501.out 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:502 2024-05-27T00:42:10.953 INFO:teuthology.orchestra.run.smithi111.stdout:502.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:503 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:503.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:504 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:504.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:505 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:505.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:506 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:506.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:507 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:507.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:508 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:508.out 2024-05-27T00:42:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:509 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:509.out 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:510 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:510.out 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:511 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:511.out 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:512 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:512.out 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:513 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:513.out 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:514 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:514.out 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:515 2024-05-27T00:42:10.955 INFO:teuthology.orchestra.run.smithi111.stdout:515.out 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:516 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:516.out 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:517 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:517.out 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:518 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:518.out 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:519 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:519.out 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:520 2024-05-27T00:42:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:520.out 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:521 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:521.out 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:522 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:522.out 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:523 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:523.out 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:524 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:524.out 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:525 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:525.out 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:526 2024-05-27T00:42:10.957 INFO:teuthology.orchestra.run.smithi111.stdout:526.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:527 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:527.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:528 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:528.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:529 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:529.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:530 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:530.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:531 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:531.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:532 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:532.out 2024-05-27T00:42:10.958 INFO:teuthology.orchestra.run.smithi111.stdout:533 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:533.out 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:534 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:534.out 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:535 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:535.out 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:536 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:536.out 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:537 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:537.out 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:538 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:538.out 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:539 2024-05-27T00:42:10.959 INFO:teuthology.orchestra.run.smithi111.stdout:539.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:540 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:540.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:541 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:541.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:542 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:542.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:543 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:543.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:544 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:544.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:545 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:545.out 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:546 2024-05-27T00:42:10.960 INFO:teuthology.orchestra.run.smithi111.stdout:546.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:547 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:547.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:548 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:548.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:549 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:549.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:550 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:550.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:551 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:551.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:552 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:552.out 2024-05-27T00:42:10.961 INFO:teuthology.orchestra.run.smithi111.stdout:553 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:553.out 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:554 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:554.out 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:555 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:555.out 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:556 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:556.out 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:557 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:557.out 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:558 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:558.out 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:559 2024-05-27T00:42:10.962 INFO:teuthology.orchestra.run.smithi111.stdout:559.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:560 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:560.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:561 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:561.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:562 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:562.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:563 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:563.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:564 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:564.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:565 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:565.out 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:566 2024-05-27T00:42:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:566.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:567 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:567.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:568 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:568.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:569 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:569.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:570 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:570.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:571 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:571.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:572 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:572.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:573 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:573.out 2024-05-27T00:42:10.964 INFO:teuthology.orchestra.run.smithi111.stdout:574 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:574.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:575 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:575.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:576 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:576.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:577 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:577.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:578 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:578.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:579 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:579.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:580 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:580.out 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:581 2024-05-27T00:42:10.965 INFO:teuthology.orchestra.run.smithi111.stdout:581.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:582 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:582.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:583 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:583.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:584 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:584.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:585 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:585.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:586 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:586.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:587 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:587.out 2024-05-27T00:42:10.966 INFO:teuthology.orchestra.run.smithi111.stdout:588 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:588.out 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:589 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:589.out 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:590 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:590.out 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:591 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:591.out 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:592 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:592.out 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:593 2024-05-27T00:42:10.967 INFO:teuthology.orchestra.run.smithi111.stdout:593.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:594 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:594.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:595 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:595.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:596 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:596.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:597 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:597.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:598 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:598.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:599 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:599.out 2024-05-27T00:42:10.968 INFO:teuthology.orchestra.run.smithi111.stdout:600 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:600.out 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:601 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:601.out 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:602 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:602.out 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:603 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:603.out 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:604 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:604.out 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:605 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:605.out 2024-05-27T00:42:10.969 INFO:teuthology.orchestra.run.smithi111.stdout:606 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:606.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:607 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:607.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:608 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:608.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:609 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:609.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:610 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:610.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:611 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:611.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:612 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:612.out 2024-05-27T00:42:10.970 INFO:teuthology.orchestra.run.smithi111.stdout:613 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:613.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:614 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:614.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:615 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:615.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:616 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:616.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:617 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:617.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:618 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:618.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:619 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:619.out 2024-05-27T00:42:10.971 INFO:teuthology.orchestra.run.smithi111.stdout:620 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:620.out 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:621 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:621.out 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:622 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:622.out 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:623 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:623.out 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:624 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:624.out 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:625 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:625.out 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:626 2024-05-27T00:42:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:626.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:627 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:627.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:628 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:628.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:629 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:629.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:630 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:630.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:631 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:631.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:632 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:632.out 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:633 2024-05-27T00:42:10.973 INFO:teuthology.orchestra.run.smithi111.stdout:633.out 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:634 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:634.out 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:635 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:635.out 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:636 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:636.out 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:637 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:637.out 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:638 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:638.out 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:639 2024-05-27T00:42:10.974 INFO:teuthology.orchestra.run.smithi111.stdout:639.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:640 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:640.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:641 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:641.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:642 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:642.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:643 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:643.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:644 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:644.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:645 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:645.out 2024-05-27T00:42:10.975 INFO:teuthology.orchestra.run.smithi111.stdout:646 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:646.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:647 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:647.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:648 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:648.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:649 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:649.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:650 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:650.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:651 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:651.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:652 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:652.out 2024-05-27T00:42:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:653 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:653.out 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:654 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:654.out 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:655 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:655.out 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:656 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:656.out 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:657 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:657.out 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:658 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:658.out 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:659 2024-05-27T00:42:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:659.out 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:660 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:660.out 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:661 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:661.out 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:662 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:662.out 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:663 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:663.out 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:664 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:664.out 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:665 2024-05-27T00:42:10.978 INFO:teuthology.orchestra.run.smithi111.stdout:665.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:666 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:666.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:667 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:667.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:668 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:668.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:669 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:669.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:670 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:670.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:671 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:671.out 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:672 2024-05-27T00:42:10.979 INFO:teuthology.orchestra.run.smithi111.stdout:672.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:673 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:673.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:674 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:674.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:675 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:675.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:676 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:676.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:677 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:677.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:678 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:678.out 2024-05-27T00:42:10.980 INFO:teuthology.orchestra.run.smithi111.stdout:679 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:679.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:680 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:680.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:681 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:681.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:682 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:682.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:683 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:683.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:684 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:684.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:685 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:685.out 2024-05-27T00:42:10.981 INFO:teuthology.orchestra.run.smithi111.stdout:686 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:686.out 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:687 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:687.out 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:688 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:688.out 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:689 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:689.out 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:690 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:690.out 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:691 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:691.out 2024-05-27T00:42:10.982 INFO:teuthology.orchestra.run.smithi111.stdout:692 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:692.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:693 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:693.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:694 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:694.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:695 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:695.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:696 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:696.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:697 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:697.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:698 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:698.out 2024-05-27T00:42:10.983 INFO:teuthology.orchestra.run.smithi111.stdout:699 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:699.out 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:700 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:700.out 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:701 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:701.out 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:702 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:702.out 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:703 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:703.out 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:704 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:704.out 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:705 2024-05-27T00:42:10.984 INFO:teuthology.orchestra.run.smithi111.stdout:705.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:706 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:706.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:707 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:707.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:708 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:708.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:709 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:709.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:710 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:710.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:711 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:711.out 2024-05-27T00:42:10.985 INFO:teuthology.orchestra.run.smithi111.stdout:712 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:712.out 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:713 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:713.out 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:714 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:714.out 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:715 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:715.out 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:716 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:716.out 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:717 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:717.out 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:718 2024-05-27T00:42:10.986 INFO:teuthology.orchestra.run.smithi111.stdout:718.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:719 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:719.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:720 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:720.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:721 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:721.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:722 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:722.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:723 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:723.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:724 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:724.out 2024-05-27T00:42:10.987 INFO:teuthology.orchestra.run.smithi111.stdout:725 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:725.out 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:726 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:726.out 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:727 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:727.out 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:728 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:728.out 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:729 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:729.out 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:730 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:730.out 2024-05-27T00:42:10.988 INFO:teuthology.orchestra.run.smithi111.stdout:731 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:731.out 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:732 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:732.out 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:733 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:733.out 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:734 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:734.out 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:735 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:735.out 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:736 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:736.out 2024-05-27T00:42:10.989 INFO:teuthology.orchestra.run.smithi111.stdout:737 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:737.out 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:738 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:738.out 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:739 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:739.out 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:740 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:740.out 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:741 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:741.out 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:742 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:742.out 2024-05-27T00:42:10.990 INFO:teuthology.orchestra.run.smithi111.stdout:743 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:743.out 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:744 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:744.out 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:745 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:745.out 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:746 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:746.out 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:747 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:747.out 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:group.list 2024-05-27T00:42:10.991 INFO:teuthology.orchestra.run.smithi111.stdout:Makefile 2024-05-27T00:42:10.993 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-05-27T00:42:11.235 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:11.239 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:11.239 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:11.240 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:16.417 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:42:16.420 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:42:16.489 INFO:teuthology.orchestra.run.smithi111.stdout:generic/001 5s 2024-05-27T00:42:16.494 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/001 2024-05-27T00:42:16.494 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:42:16.497 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:16.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:42:16.774 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-05-27T00:42:17.092 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:17.097 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:17.098 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:17.099 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:17.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:17.554 INFO:teuthology.orchestra.run.smithi111.stdout:generic/002 0s 2024-05-27T00:42:17.560 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/002 2024-05-27T00:42:17.561 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:42:17.563 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:17.602 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:42:17.603 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-05-27T00:42:17.782 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:17.786 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:17.786 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:17.787 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:17.787 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:17.992 INFO:teuthology.orchestra.run.smithi111.stdout:generic/003 [not run] atime not maintained by ceph 2024-05-27T00:42:17.996 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/003 2024-05-27T00:42:17.997 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/003 2024-05-27T00:42:17.997 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:42:17.999 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:18.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:42:18.011 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-05-27T00:42:18.187 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:18.191 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:18.191 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:18.192 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:18.192 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:18.401 INFO:teuthology.orchestra.run.smithi111.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-05-27T00:42:18.405 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/004 2024-05-27T00:42:18.405 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/004 2024-05-27T00:42:18.405 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:42:18.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:18.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:42:18.418 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-05-27T00:42:18.595 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:18.599 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:18.599 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:18.600 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:18.600 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:18.944 INFO:teuthology.orchestra.run.smithi111.stdout:generic/005 0s 2024-05-27T00:42:18.950 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/005 2024-05-27T00:42:18.950 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:42:18.953 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:18.989 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:42:18.989 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-05-27T00:42:19.169 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:19.173 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:19.173 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:19.174 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:19.175 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:43.266 INFO:teuthology.orchestra.run.smithi111.stdout:generic/006 24s 2024-05-27T00:42:43.274 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/006 2024-05-27T00:42:43.274 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:42:43.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:43.478 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:42:43.479 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-05-27T00:42:43.682 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:42:43.686 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:42:43.686 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:42:43.687 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:42:43.687 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:42:46.559 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:42:46.561 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:43:16.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:43:16.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:43:33.364 INFO:teuthology.orchestra.run.smithi111.stdout:generic/007 50s 2024-05-27T00:43:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/007 2024-05-27T00:43:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:43:33.442 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:33.482 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:43:33.483 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-05-27T00:43:33.674 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:43:33.677 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:43:33.678 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:43:33.679 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:43:33.679 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:34.026 INFO:teuthology.orchestra.run.smithi111.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-27T00:43:34.030 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/008 2024-05-27T00:43:34.030 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/008 2024-05-27T00:43:34.031 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:43:34.033 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:34.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:43:34.046 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-05-27T00:43:34.220 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:43:34.224 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:43:34.224 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:43:34.225 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:43:34.225 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:34.488 INFO:teuthology.orchestra.run.smithi111.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-27T00:43:34.491 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/009 2024-05-27T00:43:34.491 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/009 2024-05-27T00:43:34.491 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:43:34.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:34.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:43:34.508 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-05-27T00:43:34.686 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:43:34.689 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:43:34.690 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:43:34.691 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:43:34.691 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:35.282 INFO:teuthology.orchestra.run.smithi111.stdout:generic/010 1s 2024-05-27T00:43:35.423 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/010 2024-05-27T00:43:35.424 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:43:35.426 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:35.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:43:35.465 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-05-27T00:43:35.654 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:43:35.657 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:43:35.658 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:43:35.659 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:43:35.659 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:43:47.054 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:43:47.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:44:03.898 INFO:teuthology.orchestra.run.smithi111.stdout:generic/011 28s 2024-05-27T00:44:03.966 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/011 2024-05-27T00:44:03.966 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:44:03.969 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:44:04.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:44:04.012 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-05-27T00:44:04.252 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:44:04.256 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:44:04.256 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:44:04.257 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:44:04.257 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:44:04.542 INFO:teuthology.orchestra.run.smithi111.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:44:04.546 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/012 2024-05-27T00:44:04.546 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/012 2024-05-27T00:44:04.546 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:44:04.548 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:44:04.590 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:44:04.590 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-05-27T00:44:04.777 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:44:04.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:44:04.781 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:44:04.782 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:44:04.783 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:44:17.186 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:44:17.189 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:44:47.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:44:47.371 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:45:16.002 INFO:teuthology.orchestra.run.smithi111.stdout:generic/013 71s 2024-05-27T00:45:16.009 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/013 2024-05-27T00:45:16.010 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:16.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:16.032 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:16.033 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-05-27T00:45:16.280 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:16.284 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:16.285 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:16.286 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:16.286 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:16.598 INFO:teuthology.orchestra.run.smithi111.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-05-27T00:45:16.605 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/014 2024-05-27T00:45:16.605 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/014 2024-05-27T00:45:16.605 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:16.608 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:16.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:16.627 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-05-27T00:45:16.875 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:16.881 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:16.881 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:16.882 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:16.882 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:17.153 INFO:teuthology.orchestra.run.smithi111.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T00:45:17.159 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/015 2024-05-27T00:45:17.159 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/015 2024-05-27T00:45:17.159 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:17.162 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:17.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:17.196 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-05-27T00:45:17.458 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:17.463 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:17.464 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:17.465 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:17.465 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:17.839 INFO:teuthology.orchestra.run.smithi111.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:45:17.844 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/016 2024-05-27T00:45:17.844 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/016 2024-05-27T00:45:17.844 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:17.848 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:17.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:17.891 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-05-27T00:45:17.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:45:17.938 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:45:18.178 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:18.183 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:18.184 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:18.185 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:18.185 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:18.479 INFO:teuthology.orchestra.run.smithi111.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T00:45:18.484 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/017 2024-05-27T00:45:18.484 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/017 2024-05-27T00:45:18.484 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:18.487 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:18.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:18.507 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-05-27T00:45:18.752 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:18.757 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:18.757 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:18.759 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:18.759 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:19.125 INFO:teuthology.orchestra.run.smithi111.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2024-05-27T00:45:19.141 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/018 2024-05-27T00:45:19.141 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/018 2024-05-27T00:45:19.142 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:19.145 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:19.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:19.160 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-05-27T00:45:19.413 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:19.420 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:19.687 INFO:teuthology.orchestra.run.smithi111.stdout:generic/019 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:45:19.693 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/019 2024-05-27T00:45:19.693 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/019 2024-05-27T00:45:19.693 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:19.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:19.714 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-05-27T00:45:19.949 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:19.954 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:19.955 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:19.956 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:19.956 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:23.329 INFO:teuthology.orchestra.run.smithi111.stdout:generic/020 3s 2024-05-27T00:45:23.382 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/020 2024-05-27T00:45:23.383 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:23.385 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:23.424 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:23.424 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-05-27T00:45:23.702 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:23.707 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:23.707 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:23.708 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:23.709 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:24.086 INFO:teuthology.orchestra.run.smithi111.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:45:24.091 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/021 2024-05-27T00:45:24.092 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/021 2024-05-27T00:45:24.092 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:24.136 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:24.137 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-05-27T00:45:24.425 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:24.429 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:24.429 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:24.430 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:24.431 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:24.799 INFO:teuthology.orchestra.run.smithi111.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:45:24.803 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/022 2024-05-27T00:45:24.803 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/022 2024-05-27T00:45:24.803 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:24.806 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:24.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:24.847 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-05-27T00:45:25.117 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:25.122 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:25.123 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:25.124 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:25.124 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:26.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/023 1s 2024-05-27T00:45:26.398 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/023 2024-05-27T00:45:26.399 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:26.402 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:26.491 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:26.491 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-05-27T00:45:26.783 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:26.789 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:26.789 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:26.792 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:27.090 INFO:teuthology.orchestra.run.smithi111.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-05-27T00:45:27.095 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/024 2024-05-27T00:45:27.096 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/024 2024-05-27T00:45:27.096 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:27.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:27.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:27.127 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-05-27T00:45:27.415 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:27.421 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:27.421 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:27.422 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:27.423 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:27.722 INFO:teuthology.orchestra.run.smithi111.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-05-27T00:45:27.728 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/025 2024-05-27T00:45:27.728 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/025 2024-05-27T00:45:27.728 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:27.732 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:27.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:27.759 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-05-27T00:45:28.048 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:28.054 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:28.054 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:28.055 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:28.055 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:28.464 INFO:teuthology.orchestra.run.smithi111.stdout:generic/026 [not run] ceph does not define maximum ACL count 2024-05-27T00:45:28.469 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/026 2024-05-27T00:45:28.470 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/026 2024-05-27T00:45:28.470 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:28.473 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:28.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:28.494 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-05-27T00:45:28.753 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:28.758 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:28.758 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:28.760 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:28.760 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:29.031 INFO:teuthology.orchestra.run.smithi111.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T00:45:29.036 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/027 2024-05-27T00:45:29.036 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/027 2024-05-27T00:45:29.036 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:29.040 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:29.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:29.057 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-05-27T00:45:29.298 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:29.304 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:29.305 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:29.306 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:29.306 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:34.623 INFO:teuthology.orchestra.run.smithi111.stdout:generic/028 5s 2024-05-27T00:45:34.764 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/028 2024-05-27T00:45:34.764 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:34.767 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:34.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:34.788 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-05-27T00:45:35.025 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:35.029 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:35.030 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:35.031 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:35.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:38.829 INFO:teuthology.orchestra.run.smithi111.stdout:generic/029 3s 2024-05-27T00:45:38.837 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/029 2024-05-27T00:45:38.837 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:38.841 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:38.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:38.851 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-05-27T00:45:39.094 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:39.100 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:39.100 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:39.102 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:39.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:40.533 INFO:teuthology.orchestra.run.smithi111.stdout:generic/030 1s 2024-05-27T00:45:40.541 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/030 2024-05-27T00:45:40.541 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:40.544 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:40.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:40.561 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-05-27T00:45:40.792 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:40.797 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:40.798 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:40.799 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:40.799 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:41.168 INFO:teuthology.orchestra.run.smithi111.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-05-27T00:45:41.173 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/031 2024-05-27T00:45:41.173 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/031 2024-05-27T00:45:41.173 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:41.176 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:41.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:41.200 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-05-27T00:45:41.435 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:41.440 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:41.440 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:41.442 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:41.442 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:41.717 INFO:teuthology.orchestra.run.smithi111.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:45:41.722 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/032 2024-05-27T00:45:41.722 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/032 2024-05-27T00:45:41.723 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:41.725 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:41.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:41.744 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-05-27T00:45:41.985 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:41.991 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:41.991 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:41.992 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:41.993 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:42.329 INFO:teuthology.orchestra.run.smithi111.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-27T00:45:42.333 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/033 2024-05-27T00:45:42.333 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/033 2024-05-27T00:45:42.334 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:42.336 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:42.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:42.356 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-05-27T00:45:42.591 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:42.597 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:42.597 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:42.599 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:42.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:42.880 INFO:teuthology.orchestra.run.smithi111.stdout:generic/034 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:45:42.886 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/034 2024-05-27T00:45:42.886 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/034 2024-05-27T00:45:42.886 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:42.889 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:42.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:42.903 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-05-27T00:45:43.146 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:43.151 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:43.151 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:43.152 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:43.152 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:43.488 INFO:teuthology.orchestra.run.smithi111.stdout:generic/035 0s 2024-05-27T00:45:43.496 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/035 2024-05-27T00:45:43.496 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:43.499 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:43.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:43.539 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-05-27T00:45:43.828 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:43.835 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:43.835 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:43.836 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:43.837 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:48.016 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:45:48.019 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:45:54.212 INFO:teuthology.orchestra.run.smithi111.stdout:generic/036 11s 2024-05-27T00:45:54.218 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/036 2024-05-27T00:45:54.218 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:54.222 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:54.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:54.253 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-05-27T00:45:54.551 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:54.556 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:54.557 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:54.559 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:54.559 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:58.715 INFO:teuthology.orchestra.run.smithi111.stdout:generic/037 4s 2024-05-27T00:45:58.721 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/037 2024-05-27T00:45:58.721 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:58.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:58.736 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-05-27T00:45:58.973 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:58.978 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:58.978 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:58.980 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:58.980 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:59.269 INFO:teuthology.orchestra.run.smithi111.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:45:59.274 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/038 2024-05-27T00:45:59.274 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/038 2024-05-27T00:45:59.274 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:59.277 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:59.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:59.297 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-05-27T00:45:59.526 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:45:59.531 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:45:59.531 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:45:59.533 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:45:59.533 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:59.828 INFO:teuthology.orchestra.run.smithi111.stdout:generic/039 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:45:59.833 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/039 2024-05-27T00:45:59.834 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/039 2024-05-27T00:45:59.834 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:45:59.836 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:45:59.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:45:59.858 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-05-27T00:46:00.102 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:00.109 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:00.110 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:00.111 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:00.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:00.418 INFO:teuthology.orchestra.run.smithi111.stdout:generic/040 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:00.424 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/040 2024-05-27T00:46:00.424 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/040 2024-05-27T00:46:00.424 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:00.427 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:00.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:00.449 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-05-27T00:46:00.702 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:00.707 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:00.708 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:00.709 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:00.710 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:00.999 INFO:teuthology.orchestra.run.smithi111.stdout:generic/041 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:01.005 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/041 2024-05-27T00:46:01.005 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/041 2024-05-27T00:46:01.005 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:01.008 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:01.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:01.033 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-05-27T00:46:01.279 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:01.285 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:01.285 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:01.286 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:01.286 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:01.656 INFO:teuthology.orchestra.run.smithi111.stdout:generic/042 [not run] ceph does not support shutdown 2024-05-27T00:46:01.667 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/042 2024-05-27T00:46:01.667 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/042 2024-05-27T00:46:01.667 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:01.670 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:01.686 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:01.687 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-05-27T00:46:01.929 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:01.934 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:01.935 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:01.936 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:01.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:02.299 INFO:teuthology.orchestra.run.smithi111.stdout:generic/043 [not run] ceph does not support shutdown 2024-05-27T00:46:02.308 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/043 2024-05-27T00:46:02.308 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/043 2024-05-27T00:46:02.308 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:02.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:02.324 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-05-27T00:46:02.561 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:02.566 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:02.566 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:02.567 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:02.567 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:02.914 INFO:teuthology.orchestra.run.smithi111.stdout:generic/044 [not run] ceph does not support shutdown 2024-05-27T00:46:02.923 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/044 2024-05-27T00:46:02.923 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/044 2024-05-27T00:46:02.924 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:02.927 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:02.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:02.942 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-05-27T00:46:03.184 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:03.190 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:03.191 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:03.192 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:03.192 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:03.550 INFO:teuthology.orchestra.run.smithi111.stdout:generic/045 [not run] ceph does not support shutdown 2024-05-27T00:46:03.562 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/045 2024-05-27T00:46:03.562 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/045 2024-05-27T00:46:03.562 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:03.565 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:03.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:03.582 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-05-27T00:46:03.820 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:03.826 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:03.826 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:03.827 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:04.192 INFO:teuthology.orchestra.run.smithi111.stdout:generic/046 [not run] ceph does not support shutdown 2024-05-27T00:46:04.200 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/046 2024-05-27T00:46:04.200 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/046 2024-05-27T00:46:04.200 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:04.203 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:04.220 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:04.220 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-05-27T00:46:04.454 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:04.459 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:04.460 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:04.461 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:04.461 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:04.813 INFO:teuthology.orchestra.run.smithi111.stdout:generic/047 [not run] ceph does not support shutdown 2024-05-27T00:46:04.821 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/047 2024-05-27T00:46:04.821 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/047 2024-05-27T00:46:04.821 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:04.825 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:04.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:04.839 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-05-27T00:46:05.078 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:05.084 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:05.085 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:05.086 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:05.086 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:05.435 INFO:teuthology.orchestra.run.smithi111.stdout:generic/048 [not run] ceph does not support shutdown 2024-05-27T00:46:05.445 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/048 2024-05-27T00:46:05.445 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/048 2024-05-27T00:46:05.445 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:05.449 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:05.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:05.463 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-05-27T00:46:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:05.711 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:06.074 INFO:teuthology.orchestra.run.smithi111.stdout:generic/049 [not run] ceph does not support shutdown 2024-05-27T00:46:06.096 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/049 2024-05-27T00:46:06.096 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/049 2024-05-27T00:46:06.096 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:06.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:06.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:06.111 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-05-27T00:46:06.337 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:06.343 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:06.343 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:06.344 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:06.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:06.899 INFO:teuthology.orchestra.run.smithi111.stdout:generic/050 [not run] ceph does not support shutdown 2024-05-27T00:46:06.919 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/050 2024-05-27T00:46:06.920 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/050 2024-05-27T00:46:06.920 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:06.922 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:06.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:06.943 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-05-27T00:46:07.174 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:07.178 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:07.179 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:07.179 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:07.180 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:07.537 INFO:teuthology.orchestra.run.smithi111.stdout:generic/051 [not run] ceph does not support shutdown 2024-05-27T00:46:07.542 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/051 2024-05-27T00:46:07.542 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/051 2024-05-27T00:46:07.542 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:07.546 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:07.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:07.570 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-05-27T00:46:07.807 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:07.812 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:07.812 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:07.814 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:07.814 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:08.169 INFO:teuthology.orchestra.run.smithi111.stdout:generic/052 [not run] ceph does not support shutdown 2024-05-27T00:46:08.178 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/052 2024-05-27T00:46:08.178 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/052 2024-05-27T00:46:08.178 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:08.181 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:08.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:08.195 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-05-27T00:46:08.453 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:08.457 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:08.458 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:08.459 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:08.459 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:09.071 INFO:teuthology.orchestra.run.smithi111.stdout:generic/053 1s 2024-05-27T00:46:09.133 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/053 2024-05-27T00:46:09.133 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:09.138 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:09.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:09.149 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-05-27T00:46:09.390 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:09.396 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:09.396 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:09.397 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:09.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:09.786 INFO:teuthology.orchestra.run.smithi111.stdout:generic/054 [not run] ceph does not support shutdown 2024-05-27T00:46:09.797 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/054 2024-05-27T00:46:09.797 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/054 2024-05-27T00:46:09.797 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:09.801 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:09.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:09.816 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-05-27T00:46:10.057 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:10.063 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:10.063 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:10.064 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:10.064 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:10.429 INFO:teuthology.orchestra.run.smithi111.stdout:generic/055 [not run] ceph does not support shutdown 2024-05-27T00:46:10.438 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/055 2024-05-27T00:46:10.438 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/055 2024-05-27T00:46:10.438 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:10.441 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:10.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:10.458 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-05-27T00:46:10.702 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:10.709 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:10.710 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:10.710 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:10.710 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:11.013 INFO:teuthology.orchestra.run.smithi111.stdout:generic/056 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:11.019 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/056 2024-05-27T00:46:11.019 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/056 2024-05-27T00:46:11.019 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:11.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:11.045 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-05-27T00:46:11.289 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:11.295 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:11.296 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:11.297 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:11.599 INFO:teuthology.orchestra.run.smithi111.stdout:generic/057 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:11.604 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/057 2024-05-27T00:46:11.604 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/057 2024-05-27T00:46:11.605 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:11.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:11.628 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-05-27T00:46:11.876 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:11.881 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:11.882 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:11.883 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:12.257 INFO:teuthology.orchestra.run.smithi111.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:46:12.262 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/058 2024-05-27T00:46:12.262 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/058 2024-05-27T00:46:12.262 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:12.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:12.302 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-05-27T00:46:12.604 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:12.609 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:12.609 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:12.610 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:12.897 INFO:teuthology.orchestra.run.smithi111.stdout:generic/059 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:12.904 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/059 2024-05-27T00:46:12.904 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/059 2024-05-27T00:46:12.904 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:12.907 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:12.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:12.925 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-05-27T00:46:13.168 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:13.172 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:13.173 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:13.174 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:13.174 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:13.560 INFO:teuthology.orchestra.run.smithi111.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:46:13.567 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/060 2024-05-27T00:46:13.567 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/060 2024-05-27T00:46:13.567 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:13.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:13.600 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-05-27T00:46:13.912 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:13.917 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:13.917 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:13.918 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:13.919 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:14.308 INFO:teuthology.orchestra.run.smithi111.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:46:14.313 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/061 2024-05-27T00:46:14.313 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/061 2024-05-27T00:46:14.313 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:14.317 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:14.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:14.342 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-05-27T00:46:14.656 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:14.661 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:14.661 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:14.662 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:14.662 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:16.196 INFO:teuthology.orchestra.run.smithi111.stdout:generic/062 2s 2024-05-27T00:46:16.210 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/062 2024-05-27T00:46:16.210 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:16.213 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:16.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:16.231 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-05-27T00:46:16.534 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:16.540 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:16.540 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:16.541 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:16.541 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:16.976 INFO:teuthology.orchestra.run.smithi111.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:46:16.980 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/063 2024-05-27T00:46:16.980 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/063 2024-05-27T00:46:16.980 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:16.984 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:17.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:17.026 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-05-27T00:46:17.316 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:17.322 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:17.323 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:17.324 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:17.324 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:17.615 INFO:teuthology.orchestra.run.smithi111.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T00:46:17.621 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/064 2024-05-27T00:46:17.621 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/064 2024-05-27T00:46:17.621 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:17.624 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:17.642 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:17.642 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-05-27T00:46:17.880 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:17.886 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:17.887 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:17.888 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:17.888 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:18.195 INFO:teuthology.orchestra.run.smithi111.stdout:generic/065 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:18.200 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/065 2024-05-27T00:46:18.200 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/065 2024-05-27T00:46:18.200 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:18.203 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:18.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:18.223 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-05-27T00:46:18.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:46:18.269 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:46:18.471 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:18.478 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:18.479 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:18.480 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:18.481 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:18.765 INFO:teuthology.orchestra.run.smithi111.stdout:generic/066 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:18.771 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/066 2024-05-27T00:46:18.771 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/066 2024-05-27T00:46:18.771 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:18.774 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:18.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:18.789 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-05-27T00:46:19.039 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:19.044 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:19.044 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:19.045 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:19.045 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:19.349 INFO:teuthology.orchestra.run.smithi111.stdout:generic/067 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:19.354 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/067 2024-05-27T00:46:19.354 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/067 2024-05-27T00:46:19.354 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:19.357 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:19.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:19.375 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-05-27T00:46:19.620 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:19.625 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:19.626 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:19.627 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:19.922 INFO:teuthology.orchestra.run.smithi111.stdout:generic/068 [not run] ceph does not support freezing 2024-05-27T00:46:19.927 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/068 2024-05-27T00:46:19.927 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/068 2024-05-27T00:46:19.927 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:19.931 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:19.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:19.947 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-05-27T00:46:20.188 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:20.193 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:20.194 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:20.195 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:20.195 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:26.950 INFO:teuthology.orchestra.run.smithi111.stdout:generic/069 6s 2024-05-27T00:46:26.958 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/069 2024-05-27T00:46:26.958 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:26.961 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:26.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:26.974 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-05-27T00:46:27.260 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:27.265 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:27.265 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:27.266 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:27.266 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:48.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:46:48.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:46:54.634 INFO:teuthology.orchestra.run.smithi111.stdout:generic/070 27s 2024-05-27T00:46:54.640 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/070 2024-05-27T00:46:54.640 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:54.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:54.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:54.709 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-05-27T00:46:55.012 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:55.018 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:55.018 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:55.020 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:55.020 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:55.309 INFO:teuthology.orchestra.run.smithi111.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T00:46:55.315 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/071 2024-05-27T00:46:55.315 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/071 2024-05-27T00:46:55.315 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:55.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:55.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:55.337 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-05-27T00:46:55.572 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:55.578 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:55.579 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:55.580 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:55.580 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:55.935 INFO:teuthology.orchestra.run.smithi111.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-05-27T00:46:55.941 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/072 2024-05-27T00:46:55.941 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/072 2024-05-27T00:46:55.941 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:55.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:55.973 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-05-27T00:46:56.212 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:56.217 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:56.218 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:56.219 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:56.219 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:56.507 INFO:teuthology.orchestra.run.smithi111.stdout:generic/073 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:46:56.513 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/073 2024-05-27T00:46:56.513 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/073 2024-05-27T00:46:56.513 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:46:56.516 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:46:56.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:46:56.530 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-05-27T00:46:56.775 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:46:56.779 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:46:56.780 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:46:56.781 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:46:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:11.051 INFO:teuthology.orchestra.run.smithi111.stdout:generic/074 14s 2024-05-27T00:47:11.135 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/074 2024-05-27T00:47:11.135 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:12.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:12.358 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-05-27T00:47:12.704 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:12.709 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:12.709 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:12.710 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:12.710 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:18.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:47:18.540 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:47:18.593 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T00:47:18.592+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 162989) UID: 0 2024-05-27T00:47:18.593 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T00:47:18.592+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 162989) UID: 0 2024-05-27T00:47:32.880 INFO:teuthology.orchestra.run.smithi111.stdout:generic/075 20s 2024-05-27T00:47:32.887 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/075 2024-05-27T00:47:32.887 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:32.890 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:32.938 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:32.939 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-05-27T00:47:33.292 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:33.297 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:33.297 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:33.298 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:33.299 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:33.586 INFO:teuthology.orchestra.run.smithi111.stdout:generic/076 [not run] require 172.21.15.111:6789:/scratch to be local device 2024-05-27T00:47:33.590 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/076 2024-05-27T00:47:33.590 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/076 2024-05-27T00:47:33.590 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:33.593 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:33.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:33.611 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-05-27T00:47:33.876 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:33.881 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:33.881 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:33.883 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:35.064 INFO:teuthology.orchestra.run.smithi111.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T00:47:35.070 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/077 2024-05-27T00:47:35.070 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/077 2024-05-27T00:47:35.070 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:35.073 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:35.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:35.118 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-05-27T00:47:35.477 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:35.483 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:35.483 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:35.485 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:35.485 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:35.808 INFO:teuthology.orchestra.run.smithi111.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-05-27T00:47:35.813 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/078 2024-05-27T00:47:35.813 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/078 2024-05-27T00:47:35.813 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:35.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:35.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:35.853 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-05-27T00:47:36.212 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:36.218 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:36.218 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:36.220 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:36.220 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:36.651 INFO:teuthology.orchestra.run.smithi111.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-05-27T00:47:36.657 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/079 2024-05-27T00:47:36.657 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/079 2024-05-27T00:47:36.657 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:36.661 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:36.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:36.685 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-05-27T00:47:36.954 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:36.959 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:36.960 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:36.962 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:36.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:39.354 INFO:teuthology.orchestra.run.smithi111.stdout:generic/080 2s 2024-05-27T00:47:39.447 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/080 2024-05-27T00:47:39.447 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:39.451 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:39.474 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:39.475 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-05-27T00:47:39.742 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:39.749 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:39.749 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:39.751 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:39.751 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:40.089 INFO:teuthology.orchestra.run.smithi111.stdout:generic/081 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:47:40.094 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/081 2024-05-27T00:47:40.094 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/081 2024-05-27T00:47:40.095 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:40.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:40.113 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-05-27T00:47:40.381 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:40.385 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:40.386 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:40.387 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:40.387 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:40.688 INFO:teuthology.orchestra.run.smithi111.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T00:47:40.693 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/082 2024-05-27T00:47:40.693 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/082 2024-05-27T00:47:40.694 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:40.697 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:40.714 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:40.715 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-05-27T00:47:40.973 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:40.978 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:40.978 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:40.979 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:40.979 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:41.270 INFO:teuthology.orchestra.run.smithi111.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T00:47:41.276 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/083 2024-05-27T00:47:41.276 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/083 2024-05-27T00:47:41.276 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:41.280 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:41.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:41.299 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-05-27T00:47:41.557 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:41.561 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:41.562 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:41.563 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:41.563 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:47.149 INFO:teuthology.orchestra.run.smithi111.stdout:generic/084 6s 2024-05-27T00:47:47.158 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/084 2024-05-27T00:47:47.158 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:47.161 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:47.177 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:47.178 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-05-27T00:47:47.504 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:47.509 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:47.511 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:47.512 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:47.512 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:47.818 INFO:teuthology.orchestra.run.smithi111.stdout:generic/085 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:47:47.824 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/085 2024-05-27T00:47:47.824 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/085 2024-05-27T00:47:47.824 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:47.827 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:47.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:47.862 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-05-27T00:47:48.114 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:48.119 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:48.119 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:48.120 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:48.121 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:48.432 INFO:teuthology.orchestra.run.smithi111.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:47:48.437 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/086 2024-05-27T00:47:48.437 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/086 2024-05-27T00:47:48.437 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:48.441 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:48.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:48.456 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-05-27T00:47:48.749 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:48.755 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:48.756 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:48.757 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:48.757 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:49.208 INFO:teuthology.orchestra.run.smithi111.stdout:generic/087 1s 2024-05-27T00:47:49.216 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/087 2024-05-27T00:47:49.216 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:49.220 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:49.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:49.275 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-05-27T00:47:49.677 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:49.682 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:49.682 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:49.683 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:49.684 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:50.076 INFO:teuthology.orchestra.run.smithi111.stdout:generic/088 1s 2024-05-27T00:47:50.084 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/088 2024-05-27T00:47:50.084 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:47:50.087 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:47:50.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:47:50.137 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-05-27T00:47:50.517 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:47:50.522 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:47:50.522 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:47:50.524 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:47:50.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:50:24.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:50:24.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:50:54.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:50:54.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:51:24.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:51:24.706 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:51:55.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:51:55.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:51:55.044 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T00:51:55.044+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173416) UID: 0 2024-05-27T00:51:55.045 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T00:51:55.044+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173416) UID: 0 2024-05-27T00:54:49.916 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:54:49.919 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:55:20.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:55:20.056 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:55:20.158 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T00:55:20.158+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 175470) UID: 0 2024-05-27T00:55:20.159 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T00:55:20.158+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 175470) UID: 0 2024-05-27T00:56:17.497 INFO:teuthology.orchestra.run.smithi111.stdout:generic/089 507s 2024-05-27T00:56:17.695 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/089 2024-05-27T00:56:17.695 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:56:17.700 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:56:18.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:56:18.169 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-05-27T00:56:19.594 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:56:19.598 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:56:19.599 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:56:19.600 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:56:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:56:19.881 INFO:teuthology.orchestra.run.smithi111.stdout:generic/090 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:56:19.885 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/090 2024-05-27T00:56:19.885 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/090 2024-05-27T00:56:19.885 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:56:19.888 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:56:19.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:56:19.901 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-05-27T00:56:20.076 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:56:20.080 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:56:20.080 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:56:20.081 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:56:20.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:57:57.884 INFO:teuthology.orchestra.run.smithi111.stdout:generic/091 97s 2024-05-27T00:57:57.889 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/091 2024-05-27T00:57:57.889 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:57:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:57:57.939 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:57:57.940 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-05-27T00:57:58.300 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:57:58.304 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:57:58.304 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:57:58.305 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:57:58.305 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:57:58.530 INFO:teuthology.orchestra.run.smithi111.stdout:generic/092 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:57:58.534 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/092 2024-05-27T00:57:58.535 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/092 2024-05-27T00:57:58.535 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:57:58.538 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:57:58.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:57:58.552 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-05-27T00:57:58.729 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:57:58.733 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:57:58.733 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:57:58.734 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:57:58.734 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:01.053 INFO:teuthology.orchestra.run.smithi111.stdout:generic/093 3s 2024-05-27T00:58:01.060 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/093 2024-05-27T00:58:01.060 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:01.062 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:01.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:01.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-05-27T00:58:01.504 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:01.508 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:01.508 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:01.509 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:01.510 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:01.759 INFO:teuthology.orchestra.run.smithi111.stdout:generic/094 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T00:58:01.763 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/094 2024-05-27T00:58:01.763 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/094 2024-05-27T00:58:01.763 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:01.765 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:01.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:01.812 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-05-27T00:58:02.173 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:02.177 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:02.177 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:02.178 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:02.178 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:09.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:58:09.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:58:13.259 INFO:teuthology.orchestra.run.smithi111.stdout:generic/095 11s 2024-05-27T00:58:13.269 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/095 2024-05-27T00:58:13.269 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:13.276 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:13.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:13.291 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-05-27T00:58:13.514 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:13.518 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:13.519 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:13.520 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:13.520 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:13.803 INFO:teuthology.orchestra.run.smithi111.stdout:generic/096 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-27T00:58:13.807 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/096 2024-05-27T00:58:13.807 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/096 2024-05-27T00:58:13.807 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:13.809 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:13.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:13.827 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-05-27T00:58:14.016 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:14.020 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:14.020 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:14.021 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:14.644 INFO:teuthology.orchestra.run.smithi111.stdout:generic/097 0s 2024-05-27T00:58:14.650 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/097 2024-05-27T00:58:14.650 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:14.653 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:14.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:14.669 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-05-27T00:58:14.847 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:14.851 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:14.851 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:14.852 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:14.853 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:18.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/098 4s 2024-05-27T00:58:18.343 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/098 2024-05-27T00:58:18.343 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:18.346 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:18.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:18.355 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-05-27T00:58:18.537 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:18.541 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:18.541 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:18.542 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:18.542 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:19.265 INFO:teuthology.orchestra.run.smithi111.stdout:generic/099 1s 2024-05-27T00:58:19.270 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/099 2024-05-27T00:58:19.270 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:19.272 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:19.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:19.331 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-05-27T00:58:19.701 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:19.705 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:19.706 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:19.707 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:30.346 INFO:teuthology.orchestra.run.smithi111.stdout:generic/100 11s 2024-05-27T00:58:30.350 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/100 2024-05-27T00:58:30.351 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:30.353 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:33.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:33.313 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-05-27T00:58:33.612 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:33.616 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:33.616 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:33.617 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:33.617 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:33.841 INFO:teuthology.orchestra.run.smithi111.stdout:generic/101 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:58:33.845 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/101 2024-05-27T00:58:33.845 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/101 2024-05-27T00:58:33.845 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:33.847 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:33.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:33.860 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-05-27T00:58:34.205 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:34.209 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:34.209 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:34.210 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:34.210 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:34.422 INFO:teuthology.orchestra.run.smithi111.stdout:generic/102 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T00:58:34.426 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/102 2024-05-27T00:58:34.426 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/102 2024-05-27T00:58:34.426 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:34.428 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:34.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:34.440 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-05-27T00:58:34.624 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:34.628 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:34.629 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:34.630 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:34.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:34.858 INFO:teuthology.orchestra.run.smithi111.stdout:generic/103 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T00:58:34.862 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/103 2024-05-27T00:58:34.862 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/103 2024-05-27T00:58:34.862 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:34.865 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:34.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:34.908 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-05-27T00:58:35.268 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:35.272 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:35.273 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:35.274 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:35.274 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:35.500 INFO:teuthology.orchestra.run.smithi111.stdout:generic/104 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:58:35.504 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/104 2024-05-27T00:58:35.504 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/104 2024-05-27T00:58:35.504 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:35.506 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:35.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:35.522 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-05-27T00:58:35.701 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:35.704 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:35.705 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:35.706 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:35.706 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:36.148 INFO:teuthology.orchestra.run.smithi111.stdout:generic/105 1s 2024-05-27T00:58:36.159 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/105 2024-05-27T00:58:36.159 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:36.162 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:36.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:36.174 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-05-27T00:58:36.494 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:36.498 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:36.499 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:36.502 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:36.502 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:36.732 INFO:teuthology.orchestra.run.smithi111.stdout:generic/106 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:58:36.736 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/106 2024-05-27T00:58:36.736 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/106 2024-05-27T00:58:36.736 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:36.738 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:36.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:36.755 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-05-27T00:58:36.940 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:36.943 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:36.944 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:36.945 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:36.945 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:37.179 INFO:teuthology.orchestra.run.smithi111.stdout:generic/107 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:58:37.183 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/107 2024-05-27T00:58:37.183 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/107 2024-05-27T00:58:37.183 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:37.185 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:37.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:37.201 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-05-27T00:58:37.389 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:37.392 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:37.393 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:37.394 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:37.742 INFO:teuthology.orchestra.run.smithi111.stdout:generic/108 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T00:58:37.746 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/108 2024-05-27T00:58:37.746 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/108 2024-05-27T00:58:37.746 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:37.749 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:37.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:37.762 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-05-27T00:58:37.943 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:37.946 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:37.947 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:37.948 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:37.948 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:39.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:58:39.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:58:48.216 INFO:teuthology.orchestra.run.smithi111.stdout:generic/109 10s 2024-05-27T00:58:48.390 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/109 2024-05-27T00:58:48.390 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:58:48.392 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:58:48.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:58:48.401 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-05-27T00:58:48.946 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:58:48.950 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:58:48.951 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:58:48.952 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:58:48.952 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:06.126 INFO:teuthology.orchestra.run.smithi111.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T00:59:06.130 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/110 2024-05-27T00:59:06.130 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/110 2024-05-27T00:59:06.130 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:06.132 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:06.190 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:06.190 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-05-27T00:59:06.670 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:06.676 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:06.676 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:06.679 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:06.680 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:06.998 INFO:teuthology.orchestra.run.smithi111.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T00:59:07.002 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/111 2024-05-27T00:59:07.003 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/111 2024-05-27T00:59:07.003 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:07.005 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:07.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:07.062 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-05-27T00:59:07.431 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:07.465 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:07.465 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:07.465 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:07.465 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:09.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:59:09.737 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:59:21.798 INFO:teuthology.orchestra.run.smithi111.stdout:generic/112 14s 2024-05-27T00:59:21.804 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/112 2024-05-27T00:59:21.804 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:21.806 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:21.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:21.851 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-05-27T00:59:22.242 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:22.246 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:22.246 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:22.247 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:22.247 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:37.329 INFO:teuthology.orchestra.run.smithi111.stdout:generic/113 15s 2024-05-27T00:59:37.371 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/113 2024-05-27T00:59:37.371 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:37.373 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:37.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:37.434 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-05-27T00:59:37.889 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:37.894 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:37.894 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:37.896 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:37.896 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:38.211 INFO:teuthology.orchestra.run.smithi111.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-05-27T00:59:38.215 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/114 2024-05-27T00:59:38.215 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/114 2024-05-27T00:59:38.215 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:38.218 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:38.238 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:38.239 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-05-27T00:59:38.414 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:38.418 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:38.418 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:38.419 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:38.419 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:38.651 INFO:teuthology.orchestra.run.smithi111.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T00:59:38.654 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/115 2024-05-27T00:59:38.655 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/115 2024-05-27T00:59:38.655 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:38.657 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:38.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:38.710 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-05-27T00:59:39.089 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:39.093 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:39.093 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:39.094 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:39.094 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:39.338 INFO:teuthology.orchestra.run.smithi111.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T00:59:39.342 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/116 2024-05-27T00:59:39.342 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/116 2024-05-27T00:59:39.342 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:39.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:39.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:39.395 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-05-27T00:59:39.803 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:39.807 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:39.807 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:39.808 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:39.808 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:39.847 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:59:39.849 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T00:59:53.136 INFO:teuthology.orchestra.run.smithi111.stdout:generic/117 14s 2024-05-27T00:59:53.147 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/117 2024-05-27T00:59:53.147 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:53.151 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:53.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:53.164 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-05-27T00:59:53.349 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:53.353 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:53.353 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:53.354 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:53.354 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:53.896 INFO:teuthology.orchestra.run.smithi111.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T00:59:53.900 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/118 2024-05-27T00:59:53.900 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/118 2024-05-27T00:59:53.901 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:53.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:53.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:53.952 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-05-27T00:59:54.310 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:54.314 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:54.314 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:54.315 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:54.315 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:54.605 INFO:teuthology.orchestra.run.smithi111.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T00:59:54.609 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/119 2024-05-27T00:59:54.609 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/119 2024-05-27T00:59:54.609 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:54.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:54.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:54.651 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-05-27T00:59:55.017 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:55.021 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:55.021 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:55.022 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:55.022 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:55.225 INFO:teuthology.orchestra.run.smithi111.stdout:generic/120 [not run] atime not maintained by ceph 2024-05-27T00:59:55.229 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/120 2024-05-27T00:59:55.229 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/120 2024-05-27T00:59:55.229 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:55.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:55.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:55.244 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-05-27T00:59:55.433 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:55.436 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:55.437 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:55.438 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:55.674 INFO:teuthology.orchestra.run.smithi111.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T00:59:55.678 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/121 2024-05-27T00:59:55.678 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/121 2024-05-27T00:59:55.678 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:55.680 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:55.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:55.735 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-05-27T00:59:56.101 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:56.105 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:56.105 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:56.106 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:56.106 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:56.343 INFO:teuthology.orchestra.run.smithi111.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T00:59:56.347 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/122 2024-05-27T00:59:56.347 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/122 2024-05-27T00:59:56.347 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:56.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:56.398 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-05-27T00:59:56.749 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:56.753 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:56.753 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:56.754 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:57.232 INFO:teuthology.orchestra.run.smithi111.stdout:generic/123 1s 2024-05-27T00:59:57.238 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/123 2024-05-27T00:59:57.238 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:57.240 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:57.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T00:59:57.281 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-05-27T00:59:57.662 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T00:59:57.666 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T00:59:57.667 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T00:59:57.667 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T00:59:57.668 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T00:59:59.482 INFO:teuthology.orchestra.run.smithi111.stdout:generic/124 2s 2024-05-27T00:59:59.488 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/124 2024-05-27T00:59:59.488 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T00:59:59.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:00:00.041 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:00:00.042 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-05-27T01:00:00.392 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:00:00.395 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:00:00.396 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:00:00.397 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:00:00.397 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:00:10.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:00:10.086 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:00:40.177 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:00:40.181 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:01:01.835 INFO:teuthology.orchestra.run.smithi111.stdout:generic/125 61s 2024-05-27T01:01:01.840 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/125 2024-05-27T01:01:01.841 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:01:01.843 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:01:01.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:01:01.906 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-05-27T01:01:02.256 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:01:02.260 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:01:02.260 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:01:02.262 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:01:02.262 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:01:02.643 INFO:teuthology.orchestra.run.smithi111.stdout:generic/126 0s 2024-05-27T01:01:02.648 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/126 2024-05-27T01:01:02.648 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:01:02.651 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:01:02.717 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:01:02.718 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-05-27T01:01:03.123 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:01:03.127 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:01:03.127 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:01:03.128 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:01:03.128 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:01:10.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:01:10.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:01:40.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:01:40.634 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:02:10.693 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:02:10.696 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:02:40.843 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:02:40.847 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:03:10.939 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:03:10.942 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:03:41.337 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:03:41.341 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:04:11.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:04:11.411 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:04:41.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:04:41.468 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:05:11.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:05:11.598 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:05:41.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:05:41.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:06:11.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:06:11.895 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:06:41.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:06:41.965 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:07:12.243 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:07:12.246 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:07:42.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:07:42.309 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:08:12.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:08:12.649 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:08:42.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:08:42.733 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:09:12.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:09:12.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:09:25.452 INFO:teuthology.orchestra.run.smithi111.stdout:generic/127 502s 2024-05-27T01:09:25.457 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/127 2024-05-27T01:09:25.457 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:09:25.459 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:25.500 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:09:25.500 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-05-27T01:09:25.889 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:09:25.893 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:09:25.893 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:09:25.894 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:09:25.894 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:27.084 INFO:teuthology.orchestra.run.smithi111.stdout:generic/128 2s 2024-05-27T01:09:27.097 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/128 2024-05-27T01:09:27.097 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:09:27.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:27.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:09:27.118 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-05-27T01:09:27.334 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:09:27.338 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:09:27.338 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:09:27.339 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:09:27.339 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:27.589 INFO:teuthology.orchestra.run.smithi111.stdout:generic/129 [not run] Sparse files are not supported or do not work as expected 2024-05-27T01:09:27.593 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/129 2024-05-27T01:09:27.593 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/129 2024-05-27T01:09:27.594 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:09:27.596 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:27.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:09:27.618 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-05-27T01:09:27.799 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:09:27.803 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:09:27.803 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:09:27.804 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:09:27.804 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:28.025 INFO:teuthology.orchestra.run.smithi111.stdout:generic/130 [not run] Sparse files are not supported or do not work as expected 2024-05-27T01:09:28.029 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/130 2024-05-27T01:09:28.030 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/130 2024-05-27T01:09:28.030 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:09:28.032 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:28.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:09:28.053 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-05-27T01:09:28.238 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:09:28.242 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:09:28.242 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:09:28.243 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:09:28.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:29.575 INFO:teuthology.orchestra.run.smithi111.stdout:generic/131 1s 2024-05-27T01:09:29.581 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/131 2024-05-27T01:09:29.581 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:09:29.584 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:29.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:09:29.665 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-05-27T01:09:30.018 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:09:30.022 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:09:30.022 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:09:30.023 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:09:30.024 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:42.089 INFO:teuthology.orchestra.run.smithi111.stdout:generic/132 12s 2024-05-27T01:09:42.094 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/132 2024-05-27T01:09:42.094 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:09:42.096 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:42.105 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:09:42.105 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-05-27T01:09:42.282 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:09:42.287 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:09:42.287 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:09:42.288 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:09:42.288 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:09:43.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:09:43.032 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:10:13.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:10:13.115 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:10:43.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:10:43.407 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:11:13.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:11:13.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:11:43.626 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:11:43.629 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:12:13.765 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:12:13.768 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:12:43.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:12:43.890 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:13:14.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:13:14.453 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:13:19.473 INFO:teuthology.orchestra.run.smithi111.stdout:generic/133 217s 2024-05-27T01:13:19.478 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/133 2024-05-27T01:13:19.478 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:19.480 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:19.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:19.523 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-05-27T01:13:19.906 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:19.911 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:19.912 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:19.913 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:19.913 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:20.239 INFO:teuthology.orchestra.run.smithi111.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:20.243 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/134 2024-05-27T01:13:20.243 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/134 2024-05-27T01:13:20.243 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:20.245 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:20.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:20.291 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-05-27T01:13:20.668 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:20.672 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:20.672 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:20.673 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:20.674 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:21.449 INFO:teuthology.orchestra.run.smithi111.stdout:generic/135 1s 2024-05-27T01:13:21.455 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/135 2024-05-27T01:13:21.455 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:21.457 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:21.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:21.466 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-05-27T01:13:21.658 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:21.662 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:21.662 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:21.663 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:21.663 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:21.902 INFO:teuthology.orchestra.run.smithi111.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T01:13:21.906 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/136 2024-05-27T01:13:21.906 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/136 2024-05-27T01:13:21.906 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:21.909 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:21.964 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:21.965 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-05-27T01:13:22.321 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:22.325 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:22.325 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:22.326 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:22.326 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:22.566 INFO:teuthology.orchestra.run.smithi111.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:22.570 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/137 2024-05-27T01:13:22.570 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/137 2024-05-27T01:13:22.570 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:22.572 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:22.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:22.626 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-05-27T01:13:23.000 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:23.004 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:23.004 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:23.005 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:23.006 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:23.251 INFO:teuthology.orchestra.run.smithi111.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:23.255 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/138 2024-05-27T01:13:23.255 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/138 2024-05-27T01:13:23.255 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:23.257 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:23.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:23.310 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-05-27T01:13:23.687 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:23.691 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:23.692 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:23.693 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:23.693 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:23.930 INFO:teuthology.orchestra.run.smithi111.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:23.934 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/139 2024-05-27T01:13:23.934 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/139 2024-05-27T01:13:23.934 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:23.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:23.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:23.986 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-05-27T01:13:24.359 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:24.363 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:24.364 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:24.365 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:24.365 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:24.613 INFO:teuthology.orchestra.run.smithi111.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:24.617 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/140 2024-05-27T01:13:24.618 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/140 2024-05-27T01:13:24.618 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:24.620 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:24.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:24.659 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-05-27T01:13:25.040 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:25.044 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:25.044 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:25.045 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:25.046 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:25.378 INFO:teuthology.orchestra.run.smithi111.stdout:generic/141 0s 2024-05-27T01:13:25.384 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/141 2024-05-27T01:13:25.384 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:25.386 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:25.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:25.395 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-05-27T01:13:25.574 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:25.578 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:25.578 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:25.579 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:25.579 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:25.825 INFO:teuthology.orchestra.run.smithi111.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:25.829 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/142 2024-05-27T01:13:25.829 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/142 2024-05-27T01:13:25.829 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:25.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:25.891 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-05-27T01:13:26.258 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:26.262 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:26.262 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:26.263 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:26.263 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:26.514 INFO:teuthology.orchestra.run.smithi111.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:26.518 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/143 2024-05-27T01:13:26.518 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/143 2024-05-27T01:13:26.518 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:26.520 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:26.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:26.574 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-05-27T01:13:26.944 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:26.948 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:26.948 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:26.949 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:26.949 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:27.187 INFO:teuthology.orchestra.run.smithi111.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:27.191 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/144 2024-05-27T01:13:27.191 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/144 2024-05-27T01:13:27.192 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:27.194 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:27.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:27.251 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-05-27T01:13:27.609 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:27.613 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:27.614 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:27.615 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:27.615 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:27.853 INFO:teuthology.orchestra.run.smithi111.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:27.857 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/145 2024-05-27T01:13:27.857 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/145 2024-05-27T01:13:27.857 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:27.859 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:27.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:27.911 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-05-27T01:13:28.268 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:28.272 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:28.272 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:28.273 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:28.273 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:28.517 INFO:teuthology.orchestra.run.smithi111.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:28.521 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/146 2024-05-27T01:13:28.521 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/146 2024-05-27T01:13:28.521 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:28.523 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:28.574 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:28.574 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-05-27T01:13:28.941 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:28.945 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:28.945 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:28.946 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:28.946 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:29.188 INFO:teuthology.orchestra.run.smithi111.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:29.192 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/147 2024-05-27T01:13:29.192 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/147 2024-05-27T01:13:29.193 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:29.195 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:29.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:29.243 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-05-27T01:13:29.611 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:29.615 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:29.615 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:29.616 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:29.616 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:29.865 INFO:teuthology.orchestra.run.smithi111.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:29.869 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/148 2024-05-27T01:13:29.869 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/148 2024-05-27T01:13:29.869 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:29.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:29.922 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:29.922 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-05-27T01:13:30.294 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:30.298 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:30.298 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:30.299 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:30.300 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:30.536 INFO:teuthology.orchestra.run.smithi111.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:30.540 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/149 2024-05-27T01:13:30.540 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/149 2024-05-27T01:13:30.540 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:30.543 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:30.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:30.598 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-05-27T01:13:30.961 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:30.965 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:30.965 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:30.967 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:30.967 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:31.206 INFO:teuthology.orchestra.run.smithi111.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:31.210 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/150 2024-05-27T01:13:31.210 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/150 2024-05-27T01:13:31.211 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:31.213 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:31.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:31.261 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-05-27T01:13:31.618 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:31.621 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:31.622 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:31.623 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:31.623 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:31.864 INFO:teuthology.orchestra.run.smithi111.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:31.868 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/151 2024-05-27T01:13:31.868 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/151 2024-05-27T01:13:31.868 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:31.870 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:31.926 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:31.927 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-05-27T01:13:32.303 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:32.306 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:32.307 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:32.308 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:32.308 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:32.558 INFO:teuthology.orchestra.run.smithi111.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:32.562 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/152 2024-05-27T01:13:32.562 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/152 2024-05-27T01:13:32.562 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:32.564 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:32.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:32.621 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-05-27T01:13:32.977 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:32.981 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:32.982 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:32.983 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:32.983 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:33.220 INFO:teuthology.orchestra.run.smithi111.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:33.224 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/153 2024-05-27T01:13:33.225 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/153 2024-05-27T01:13:33.225 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:33.227 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:33.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:33.285 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-05-27T01:13:33.655 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:33.659 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:33.659 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:33.660 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:33.660 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:33.906 INFO:teuthology.orchestra.run.smithi111.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:33.910 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/154 2024-05-27T01:13:33.910 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/154 2024-05-27T01:13:33.910 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:33.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:33.964 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-05-27T01:13:34.316 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:34.320 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:34.320 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:34.321 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:34.321 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:34.553 INFO:teuthology.orchestra.run.smithi111.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:34.557 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/155 2024-05-27T01:13:34.557 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/155 2024-05-27T01:13:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:34.560 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:34.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:34.606 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-05-27T01:13:34.982 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:34.986 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:34.987 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:34.988 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:34.988 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:35.227 INFO:teuthology.orchestra.run.smithi111.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:35.231 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/156 2024-05-27T01:13:35.231 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/156 2024-05-27T01:13:35.231 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:35.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:35.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:35.284 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-05-27T01:13:35.660 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:35.664 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:35.665 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:35.666 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:35.666 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:35.904 INFO:teuthology.orchestra.run.smithi111.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:35.907 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/157 2024-05-27T01:13:35.907 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/157 2024-05-27T01:13:35.907 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:35.910 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:35.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:35.962 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-05-27T01:13:36.337 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:36.341 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:36.341 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:36.342 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:36.343 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:36.584 INFO:teuthology.orchestra.run.smithi111.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T01:13:36.587 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/158 2024-05-27T01:13:36.588 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/158 2024-05-27T01:13:36.588 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:36.590 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:36.638 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:36.639 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-05-27T01:13:37.007 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:37.011 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:37.011 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:37.012 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:37.012 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:37.272 INFO:teuthology.orchestra.run.smithi111.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-05-27T01:13:37.275 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/159 2024-05-27T01:13:37.275 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/159 2024-05-27T01:13:37.276 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:37.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:37.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:37.297 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-05-27T01:13:37.470 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:37.474 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:37.474 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:37.475 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:37.711 INFO:teuthology.orchestra.run.smithi111.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-05-27T01:13:37.715 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/160 2024-05-27T01:13:37.715 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/160 2024-05-27T01:13:37.716 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:37.718 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:37.736 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:37.737 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-05-27T01:13:37.920 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:37.924 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:37.924 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:37.925 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:37.925 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:38.270 INFO:teuthology.orchestra.run.smithi111.stdout:generic/161 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:38.277 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/161 2024-05-27T01:13:38.277 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/161 2024-05-27T01:13:38.277 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:38.280 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:38.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:38.524 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-05-27T01:13:38.725 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:38.729 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:38.730 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:38.731 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:38.731 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:39.040 INFO:teuthology.orchestra.run.smithi111.stdout:generic/162 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-05-27T01:13:39.078 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/162 2024-05-27T01:13:39.078 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/162 2024-05-27T01:13:39.078 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:39.080 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:39.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:39.395 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-05-27T01:13:39.582 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:39.586 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:39.587 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:39.588 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:39.588 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:39.903 INFO:teuthology.orchestra.run.smithi111.stdout:generic/163 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-05-27T01:13:39.944 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/163 2024-05-27T01:13:39.944 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/163 2024-05-27T01:13:39.944 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:39.946 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:40.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:40.261 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-05-27T01:13:40.451 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:40.455 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:40.455 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:40.456 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:40.457 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:40.815 INFO:teuthology.orchestra.run.smithi111.stdout:generic/164 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:40.822 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/164 2024-05-27T01:13:40.822 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/164 2024-05-27T01:13:40.822 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:40.829 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:41.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:41.002 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-05-27T01:13:41.241 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:41.245 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:41.246 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:41.247 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:41.247 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:41.595 INFO:teuthology.orchestra.run.smithi111.stdout:generic/165 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:41.602 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/165 2024-05-27T01:13:41.602 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/165 2024-05-27T01:13:41.603 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:41.605 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:41.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:41.835 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-05-27T01:13:42.047 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:42.051 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:42.052 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:42.053 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:42.053 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:42.398 INFO:teuthology.orchestra.run.smithi111.stdout:generic/166 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:42.404 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/166 2024-05-27T01:13:42.404 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/166 2024-05-27T01:13:42.404 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:42.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:42.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:42.650 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-05-27T01:13:42.867 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:42.871 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:42.871 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:42.872 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:42.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:43.225 INFO:teuthology.orchestra.run.smithi111.stdout:generic/167 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:43.238 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/167 2024-05-27T01:13:43.238 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/167 2024-05-27T01:13:43.238 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:43.242 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:43.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:43.484 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-05-27T01:13:43.680 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:43.684 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:43.685 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:43.686 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:43.686 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:44.038 INFO:teuthology.orchestra.run.smithi111.stdout:generic/168 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:44.047 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/168 2024-05-27T01:13:44.047 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/168 2024-05-27T01:13:44.047 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:44.052 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:44.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:44.275 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-05-27T01:13:44.498 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:44.502 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:44.502 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:44.503 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:44.503 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:44.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:13:44.508 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:13:44.955 INFO:teuthology.orchestra.run.smithi111.stdout:generic/169 0s 2024-05-27T01:13:44.960 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/169 2024-05-27T01:13:44.960 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:44.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:44.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:44.972 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-05-27T01:13:45.195 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:45.199 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:45.199 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:45.200 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:45.200 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:45.562 INFO:teuthology.orchestra.run.smithi111.stdout:generic/170 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:45.568 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/170 2024-05-27T01:13:45.571 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/170 2024-05-27T01:13:45.571 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:45.572 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:45.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:45.818 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-05-27T01:13:46.018 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:46.022 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:46.022 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:46.023 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:46.023 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:46.406 INFO:teuthology.orchestra.run.smithi111.stdout:generic/171 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:46.412 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/171 2024-05-27T01:13:46.412 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/171 2024-05-27T01:13:46.412 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:46.416 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:46.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:46.661 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-05-27T01:13:46.858 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:46.862 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:46.863 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:46.864 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:46.864 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:47.213 INFO:teuthology.orchestra.run.smithi111.stdout:generic/172 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:47.224 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/172 2024-05-27T01:13:47.224 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/172 2024-05-27T01:13:47.224 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:47.226 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:47.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:47.468 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-05-27T01:13:47.676 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:47.680 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:47.680 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:47.681 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:47.681 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:48.056 INFO:teuthology.orchestra.run.smithi111.stdout:generic/173 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:48.063 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/173 2024-05-27T01:13:48.063 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/173 2024-05-27T01:13:48.063 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:48.065 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:48.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:48.272 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-05-27T01:13:48.493 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:48.497 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:48.497 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:48.498 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:48.498 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:48.868 INFO:teuthology.orchestra.run.smithi111.stdout:generic/174 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:48.875 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/174 2024-05-27T01:13:48.875 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/174 2024-05-27T01:13:48.876 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:48.878 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:49.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:49.168 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-05-27T01:13:49.359 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:49.363 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:49.364 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:49.365 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:49.365 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:49.721 INFO:teuthology.orchestra.run.smithi111.stdout:generic/175 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:49.728 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/175 2024-05-27T01:13:49.728 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/175 2024-05-27T01:13:49.728 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:49.730 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:50.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:50.006 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-05-27T01:13:50.192 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:50.196 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:50.196 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:50.197 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:50.198 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:50.540 INFO:teuthology.orchestra.run.smithi111.stdout:generic/176 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/176 2024-05-27T01:13:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/176 2024-05-27T01:13:50.547 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:50.549 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:50.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:50.809 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-05-27T01:13:51.008 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:51.012 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:51.013 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:51.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:51.389 INFO:teuthology.orchestra.run.smithi111.stdout:generic/177 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T01:13:51.393 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/177 2024-05-27T01:13:51.393 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/177 2024-05-27T01:13:51.393 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:51.396 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:51.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:51.449 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-05-27T01:13:51.825 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:51.830 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:51.830 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:51.831 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:51.831 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:52.075 INFO:teuthology.orchestra.run.smithi111.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:52.079 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/178 2024-05-27T01:13:52.079 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/178 2024-05-27T01:13:52.079 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:52.082 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:52.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:52.137 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-05-27T01:13:52.503 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:52.507 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:52.507 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:52.508 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:52.508 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:52.754 INFO:teuthology.orchestra.run.smithi111.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:52.757 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/179 2024-05-27T01:13:52.757 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/179 2024-05-27T01:13:52.757 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:52.759 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:52.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:52.813 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-05-27T01:13:53.204 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:53.208 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:53.208 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:53.209 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:53.209 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:53.452 INFO:teuthology.orchestra.run.smithi111.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:53.456 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/180 2024-05-27T01:13:53.456 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/180 2024-05-27T01:13:53.457 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:53.459 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:53.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:53.512 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-05-27T01:13:53.895 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:53.899 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:53.900 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:53.901 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:53.901 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:54.138 INFO:teuthology.orchestra.run.smithi111.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:13:54.142 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/181 2024-05-27T01:13:54.142 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/181 2024-05-27T01:13:54.142 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:54.145 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:54.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:54.196 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-05-27T01:13:54.569 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:54.573 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:54.574 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:54.575 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:54.575 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:54.814 INFO:teuthology.orchestra.run.smithi111.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T01:13:54.818 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/182 2024-05-27T01:13:54.818 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/182 2024-05-27T01:13:54.818 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:54.821 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:54.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:54.877 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-05-27T01:13:55.244 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:55.248 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:55.248 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:55.249 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:55.249 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:55.595 INFO:teuthology.orchestra.run.smithi111.stdout:generic/183 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:55.602 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/183 2024-05-27T01:13:55.602 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/183 2024-05-27T01:13:55.602 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:55.604 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:55.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:55.873 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-05-27T01:13:56.068 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:56.072 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:56.072 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:56.073 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:56.074 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:56.530 INFO:teuthology.orchestra.run.smithi111.stdout:generic/184 0s 2024-05-27T01:13:56.536 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/184 2024-05-27T01:13:56.536 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:56.538 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:56.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:56.557 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-05-27T01:13:56.734 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:56.737 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:56.738 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:56.739 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:56.739 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:57.077 INFO:teuthology.orchestra.run.smithi111.stdout:generic/185 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:57.083 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/185 2024-05-27T01:13:57.083 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/185 2024-05-27T01:13:57.083 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:57.091 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:57.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:57.342 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-05-27T01:13:57.543 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:57.547 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:57.548 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:57.549 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:57.549 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:57.905 INFO:teuthology.orchestra.run.smithi111.stdout:generic/186 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:57.912 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/186 2024-05-27T01:13:57.912 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/186 2024-05-27T01:13:57.913 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:57.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:58.141 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:58.141 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-05-27T01:13:58.359 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:58.363 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:58.363 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:58.364 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:58.733 INFO:teuthology.orchestra.run.smithi111.stdout:generic/187 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:58.738 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/187 2024-05-27T01:13:58.739 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/187 2024-05-27T01:13:58.739 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:58.742 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:58.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:58.997 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-05-27T01:13:59.193 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:13:59.197 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:13:59.198 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:13:59.199 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:13:59.199 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:59.561 INFO:teuthology.orchestra.run.smithi111.stdout:generic/188 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:13:59.566 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/188 2024-05-27T01:13:59.567 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/188 2024-05-27T01:13:59.567 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:13:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:13:59.820 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:13:59.820 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-05-27T01:14:00.026 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:00.030 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:00.030 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:00.031 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:00.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:00.397 INFO:teuthology.orchestra.run.smithi111.stdout:generic/189 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:00.407 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/189 2024-05-27T01:14:00.407 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/189 2024-05-27T01:14:00.407 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:00.413 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:00.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:00.661 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-05-27T01:14:00.871 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:00.875 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:00.875 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:00.876 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:00.876 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:01.241 INFO:teuthology.orchestra.run.smithi111.stdout:generic/190 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:01.246 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/190 2024-05-27T01:14:01.246 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/190 2024-05-27T01:14:01.247 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:01.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:01.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:01.492 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-05-27T01:14:01.702 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:01.706 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:01.707 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:01.708 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:01.708 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:02.062 INFO:teuthology.orchestra.run.smithi111.stdout:generic/191 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:02.068 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/191 2024-05-27T01:14:02.068 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/191 2024-05-27T01:14:02.068 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:02.072 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:02.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:02.339 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-05-27T01:14:02.541 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:02.545 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:02.545 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:02.546 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:02.546 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:02.769 INFO:teuthology.orchestra.run.smithi111.stdout:generic/192 [not run] atime not maintained by ceph 2024-05-27T01:14:02.773 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/192 2024-05-27T01:14:02.773 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/192 2024-05-27T01:14:02.773 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:02.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:02.788 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-05-27T01:14:02.972 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:02.976 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:02.977 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:02.978 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:02.978 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:04.059 INFO:teuthology.orchestra.run.smithi111.stdout:generic/193 1s 2024-05-27T01:14:04.064 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/193 2024-05-27T01:14:04.064 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:04.067 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:04.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:04.121 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-05-27T01:14:04.501 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:04.505 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:04.506 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:04.506 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:04.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:04.869 INFO:teuthology.orchestra.run.smithi111.stdout:generic/194 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:04.877 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/194 2024-05-27T01:14:04.877 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/194 2024-05-27T01:14:04.877 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:04.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:05.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:05.144 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-05-27T01:14:05.346 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:05.350 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:05.350 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:05.351 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:05.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:generic/195 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/195 2024-05-27T01:14:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/195 2024-05-27T01:14:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:05.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:05.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:05.996 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-05-27T01:14:06.198 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:06.202 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:06.202 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:06.203 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:06.203 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:06.545 INFO:teuthology.orchestra.run.smithi111.stdout:generic/196 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:06.551 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/196 2024-05-27T01:14:06.551 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/196 2024-05-27T01:14:06.551 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:06.559 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:06.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:06.827 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-05-27T01:14:07.033 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:07.037 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:07.037 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:07.038 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:07.038 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:07.381 INFO:teuthology.orchestra.run.smithi111.stdout:generic/197 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:07.388 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/197 2024-05-27T01:14:07.388 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/197 2024-05-27T01:14:07.388 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:07.392 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:07.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:07.665 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-05-27T01:14:07.864 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:07.868 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:07.868 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:07.869 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:07.869 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:08.582 INFO:teuthology.orchestra.run.smithi111.stdout:generic/198 1s 2024-05-27T01:14:08.587 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/198 2024-05-27T01:14:08.587 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:08.589 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:08.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:08.634 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-05-27T01:14:09.043 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:09.049 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:09.050 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:09.051 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:09.052 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:09.423 INFO:teuthology.orchestra.run.smithi111.stdout:generic/199 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:09.430 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/199 2024-05-27T01:14:09.430 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/199 2024-05-27T01:14:09.430 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:09.433 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:09.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:09.693 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-05-27T01:14:09.904 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:09.908 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:09.908 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:09.909 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:09.909 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:10.279 INFO:teuthology.orchestra.run.smithi111.stdout:generic/200 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:10.286 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/200 2024-05-27T01:14:10.286 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/200 2024-05-27T01:14:10.286 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:10.288 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:10.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:10.517 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-05-27T01:14:10.731 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:10.735 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:10.735 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:10.736 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:10.736 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:11.091 INFO:teuthology.orchestra.run.smithi111.stdout:generic/201 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:11.100 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/201 2024-05-27T01:14:11.100 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/201 2024-05-27T01:14:11.100 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:11.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:11.348 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-05-27T01:14:11.559 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:11.563 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:11.563 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:11.565 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:11.922 INFO:teuthology.orchestra.run.smithi111.stdout:generic/202 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:11.929 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/202 2024-05-27T01:14:11.929 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/202 2024-05-27T01:14:11.929 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:12.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:12.181 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-05-27T01:14:12.389 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:12.392 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:12.393 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:12.394 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:12.770 INFO:teuthology.orchestra.run.smithi111.stdout:generic/203 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:12.777 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/203 2024-05-27T01:14:12.777 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/203 2024-05-27T01:14:12.777 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:12.779 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:13.042 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:13.043 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-05-27T01:14:13.244 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:13.248 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:13.248 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:13.249 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:13.249 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:14.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:14:14.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:14:15.338 INFO:teuthology.orchestra.run.smithi111.stdout:generic/204 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:14:15.343 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/204 2024-05-27T01:14:15.343 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/204 2024-05-27T01:14:15.344 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:15.347 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:15.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:15.364 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-05-27T01:14:15.551 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:15.555 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:15.556 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:15.557 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:15.557 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:15.926 INFO:teuthology.orchestra.run.smithi111.stdout:generic/205 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:15.932 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/205 2024-05-27T01:14:15.932 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/205 2024-05-27T01:14:15.932 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:15.939 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:16.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:16.223 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-05-27T01:14:16.427 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:16.431 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:16.431 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:16.432 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:16.433 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:16.810 INFO:teuthology.orchestra.run.smithi111.stdout:generic/206 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:14:16.817 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/206 2024-05-27T01:14:16.818 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/206 2024-05-27T01:14:16.818 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:16.819 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:17.091 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:17.108 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-05-27T01:14:17.334 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:17.338 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:17.339 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:17.339 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:17.340 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:23.964 INFO:teuthology.orchestra.run.smithi111.stdout:generic/207 6s 2024-05-27T01:14:23.969 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/207 2024-05-27T01:14:23.969 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:14:23.972 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:24.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:14:24.014 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-05-27T01:14:24.390 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:14:24.394 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:14:24.394 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:14:24.395 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:14:24.395 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:14:45.102 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:14:45.105 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:15:15.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:15:15.169 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:15:45.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:15:45.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:16:15.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:16:15.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:16:15.651 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T01:16:15.651+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 249785) UID: 0 2024-05-27T01:16:15.652 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T01:16:15.651+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 249785) UID: 0 2024-05-27T01:16:15.652 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T01:16:15.651+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 249785) UID: 0 2024-05-27T01:16:15.653 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T01:16:15.651+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 249785) UID: 0 2024-05-27T01:16:15.661 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T01:16:15.660+0000 7f6764b08640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49067) UID: 0 2024-05-27T01:16:15.661 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T01:16:15.660+0000 7f76dfbad640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49067) UID: 0 2024-05-27T01:16:15.661 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T01:16:15.661+0000 7f2307fdb640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49067) UID: 0 2024-05-27T01:16:15.661 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T01:16:15.660+0000 7f62b9de8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49067) UID: 0 2024-05-27T01:17:44.723 INFO:teuthology.orchestra.run.smithi111.stdout:generic/208 200s 2024-05-27T01:17:44.728 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/208 2024-05-27T01:17:44.728 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:17:44.730 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:17:44.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:17:44.776 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-05-27T01:17:45.219 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:17:45.222 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:17:45.223 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:17:45.224 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:17:45.224 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:17.270 INFO:teuthology.orchestra.run.smithi111.stdout:generic/209 32s 2024-05-27T01:18:17.275 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/209 2024-05-27T01:18:17.275 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:17.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:17.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:17.327 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-05-27T01:18:17.734 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:17.738 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:17.738 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:17.739 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:17.739 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:18.070 INFO:teuthology.orchestra.run.smithi111.stdout:generic/210 1s 2024-05-27T01:18:18.075 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/210 2024-05-27T01:18:18.076 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:18.078 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:18.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:18.126 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-05-27T01:18:18.538 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:18.543 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:18.543 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:18.544 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:18.545 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:18.973 INFO:teuthology.orchestra.run.smithi111.stdout:generic/211 0s 2024-05-27T01:18:18.978 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/211 2024-05-27T01:18:18.978 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:18.981 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:18.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:18.994 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-05-27T01:18:19.177 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:19.180 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:19.181 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:19.182 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:19.182 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:19.466 INFO:teuthology.orchestra.run.smithi111.stdout:generic/212 0s 2024-05-27T01:18:19.472 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/212 2024-05-27T01:18:19.472 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:19.474 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:19.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:19.493 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-05-27T01:18:19.680 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:19.684 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:19.685 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:19.686 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:19.929 INFO:teuthology.orchestra.run.smithi111.stdout:generic/213 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:18:19.932 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/213 2024-05-27T01:18:19.933 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/213 2024-05-27T01:18:19.933 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:19.935 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:19.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:19.949 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-05-27T01:18:20.126 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:20.130 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:20.131 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:20.132 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:20.132 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:20.360 INFO:teuthology.orchestra.run.smithi111.stdout:generic/214 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:18:20.364 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/214 2024-05-27T01:18:20.364 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/214 2024-05-27T01:18:20.364 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:20.366 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:20.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:20.382 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-05-27T01:18:20.566 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:20.570 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:20.570 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:20.571 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:20.572 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:23.301 INFO:teuthology.orchestra.run.smithi111.stdout:generic/215 3s 2024-05-27T01:18:23.306 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/215 2024-05-27T01:18:23.307 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:23.309 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:23.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:23.341 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-05-27T01:18:23.527 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:23.534 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:23.534 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:23.535 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:23.535 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:23.894 INFO:teuthology.orchestra.run.smithi111.stdout:generic/216 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:23.901 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/216 2024-05-27T01:18:23.901 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/216 2024-05-27T01:18:23.901 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:23.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:24.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:24.201 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-05-27T01:18:24.408 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:24.412 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:24.412 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:24.414 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:24.414 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:24.781 INFO:teuthology.orchestra.run.smithi111.stdout:generic/217 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:24.788 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/217 2024-05-27T01:18:24.788 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/217 2024-05-27T01:18:24.788 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:24.791 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:25.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:25.082 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-05-27T01:18:25.301 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:25.305 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:25.306 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:25.307 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:25.307 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:25.673 INFO:teuthology.orchestra.run.smithi111.stdout:generic/218 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:25.680 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/218 2024-05-27T01:18:25.680 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/218 2024-05-27T01:18:25.680 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:25.682 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:25.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:25.967 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-05-27T01:18:26.200 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:26.204 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:26.204 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:26.206 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:26.206 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:26.424 INFO:teuthology.orchestra.run.smithi111.stdout:generic/219 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:26.428 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/219 2024-05-27T01:18:26.428 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/219 2024-05-27T01:18:26.428 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:26.430 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:26.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:26.445 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-05-27T01:18:26.622 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:26.625 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:26.626 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:26.627 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:26.627 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:26.999 INFO:teuthology.orchestra.run.smithi111.stdout:generic/220 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:27.004 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/220 2024-05-27T01:18:27.005 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/220 2024-05-27T01:18:27.005 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:27.008 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:27.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:27.327 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-05-27T01:18:27.524 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:27.527 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:27.528 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:27.529 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:27.529 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:28.786 INFO:teuthology.orchestra.run.smithi111.stdout:generic/221 1s 2024-05-27T01:18:28.791 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/221 2024-05-27T01:18:28.791 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:28.793 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:28.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:28.813 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-05-27T01:18:28.993 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:28.997 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:28.998 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:28.999 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:28.999 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:29.345 INFO:teuthology.orchestra.run.smithi111.stdout:generic/222 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:29.350 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/222 2024-05-27T01:18:29.351 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/222 2024-05-27T01:18:29.351 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:29.353 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:29.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:29.666 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-05-27T01:18:29.873 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:29.878 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:29.878 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:29.879 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:29.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:30.112 INFO:teuthology.orchestra.run.smithi111.stdout:generic/223 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:18:30.116 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/223 2024-05-27T01:18:30.116 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/223 2024-05-27T01:18:30.116 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:30.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:30.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:30.136 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-05-27T01:18:30.315 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:30.319 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:30.319 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:30.321 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:30.321 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:30.537 INFO:teuthology.orchestra.run.smithi111.stdout:generic/224 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:18:30.541 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/224 2024-05-27T01:18:30.541 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/224 2024-05-27T01:18:30.541 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:30.544 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:30.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:30.557 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-05-27T01:18:30.738 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:30.742 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:30.742 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:30.743 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:30.744 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:31.013 INFO:teuthology.orchestra.run.smithi111.stdout:generic/225 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T01:18:31.017 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/225 2024-05-27T01:18:31.017 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/225 2024-05-27T01:18:31.017 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:31.019 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:31.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:31.065 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-05-27T01:18:31.504 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:31.508 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:31.508 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:31.509 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:31.509 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:31.755 INFO:teuthology.orchestra.run.smithi111.stdout:generic/226 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:18:31.759 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/226 2024-05-27T01:18:31.760 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/226 2024-05-27T01:18:31.760 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:31.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:31.781 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:31.781 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-05-27T01:18:31.958 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:31.961 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:31.962 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:31.963 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:31.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:32.330 INFO:teuthology.orchestra.run.smithi111.stdout:generic/227 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:32.335 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/227 2024-05-27T01:18:32.335 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/227 2024-05-27T01:18:32.335 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:32.342 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:32.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:32.685 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-05-27T01:18:32.883 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:32.886 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:32.887 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:32.888 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:32.888 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:33.146 INFO:teuthology.orchestra.run.smithi111.stdout:generic/228 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:18:33.150 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/228 2024-05-27T01:18:33.150 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/228 2024-05-27T01:18:33.150 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:33.152 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:33.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:33.166 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-05-27T01:18:33.350 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:33.354 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:33.354 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:33.355 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:33.355 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:33.699 INFO:teuthology.orchestra.run.smithi111.stdout:generic/229 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:33.709 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/229 2024-05-27T01:18:33.709 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/229 2024-05-27T01:18:33.709 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:33.715 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:33.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:33.995 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-05-27T01:18:34.229 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:34.233 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:34.234 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:34.235 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:34.235 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:34.448 INFO:teuthology.orchestra.run.smithi111.stdout:generic/230 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:34.452 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/230 2024-05-27T01:18:34.453 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/230 2024-05-27T01:18:34.453 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:34.455 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:34.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:34.483 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-05-27T01:18:34.658 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:34.662 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:34.663 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:34.664 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:34.664 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:34.878 INFO:teuthology.orchestra.run.smithi111.stdout:generic/231 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:34.882 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/231 2024-05-27T01:18:34.882 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/231 2024-05-27T01:18:34.882 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:34.884 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:34.898 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:34.898 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-05-27T01:18:35.079 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:35.083 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:35.083 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:35.084 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:35.084 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:35.298 INFO:teuthology.orchestra.run.smithi111.stdout:generic/232 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:35.302 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/232 2024-05-27T01:18:35.302 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/232 2024-05-27T01:18:35.302 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:35.304 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:35.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:35.329 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-05-27T01:18:35.512 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:35.516 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:35.517 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:35.519 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:35.519 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:35.728 INFO:teuthology.orchestra.run.smithi111.stdout:generic/233 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:35.732 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/233 2024-05-27T01:18:35.732 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/233 2024-05-27T01:18:35.732 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:35.735 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:35.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:35.748 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-05-27T01:18:35.935 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:35.939 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:35.939 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:35.940 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:35.940 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:36.149 INFO:teuthology.orchestra.run.smithi111.stdout:generic/234 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:36.153 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/234 2024-05-27T01:18:36.153 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/234 2024-05-27T01:18:36.153 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:36.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:36.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:36.169 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-05-27T01:18:36.346 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:36.349 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:36.350 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:36.351 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:36.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:36.561 INFO:teuthology.orchestra.run.smithi111.stdout:generic/235 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:18:36.565 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/235 2024-05-27T01:18:36.565 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/235 2024-05-27T01:18:36.566 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:36.568 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:36.581 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:36.581 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-05-27T01:18:36.766 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:36.770 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:36.771 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:36.772 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:36.772 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:38.058 INFO:teuthology.orchestra.run.smithi111.stdout:generic/236 2s 2024-05-27T01:18:38.064 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/236 2024-05-27T01:18:38.064 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:38.066 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:38.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:38.118 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-05-27T01:18:38.558 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:38.562 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:38.563 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:38.564 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:38.564 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:38.882 INFO:teuthology.orchestra.run.smithi111.stdout:generic/237 0s 2024-05-27T01:18:38.887 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/237 2024-05-27T01:18:38.888 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:38.890 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:38.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:38.942 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-05-27T01:18:39.366 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:39.370 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:39.370 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:39.371 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:39.372 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:39.748 INFO:teuthology.orchestra.run.smithi111.stdout:generic/238 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:18:39.755 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/238 2024-05-27T01:18:39.755 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/238 2024-05-27T01:18:39.755 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:39.757 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:40.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:40.092 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-05-27T01:18:40.275 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:40.279 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:40.279 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:40.280 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:40.280 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:40.541 INFO:teuthology.orchestra.run.smithi111.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-05-27T01:18:40.544 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/239 2024-05-27T01:18:40.544 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/239 2024-05-27T01:18:40.545 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:40.547 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:40.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:40.571 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-05-27T01:18:40.749 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:40.753 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:40.754 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:40.755 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:40.755 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:40.992 INFO:teuthology.orchestra.run.smithi111.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-05-27T01:18:40.996 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/240 2024-05-27T01:18:40.996 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/240 2024-05-27T01:18:40.996 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:18:40.999 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:18:41.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:18:41.024 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-05-27T01:18:41.210 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:18:41.214 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:18:41.215 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:18:41.216 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:18:41.216 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:24.710 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:19:24.714 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:19:24.744 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T01:19:24.744+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 264981) UID: 0 2024-05-27T01:19:24.744 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T01:19:24.744+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 264981) UID: 0 2024-05-27T01:19:24.745 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T01:19:24.744+0000 7f6764b08640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49132) UID: 0 2024-05-27T01:19:24.745 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T01:19:24.744+0000 7f76dfbad640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49132) UID: 0 2024-05-27T01:19:24.745 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T01:19:24.744+0000 7f2307fdb640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49132) UID: 0 2024-05-27T01:19:24.746 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T01:19:24.744+0000 7f62b9de8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49132) UID: 0 2024-05-27T01:19:54.445 INFO:teuthology.orchestra.run.smithi111.stdout:generic/241 73s 2024-05-27T01:19:54.451 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/241 2024-05-27T01:19:54.451 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:19:54.454 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:56.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:19:56.264 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-05-27T01:19:56.719 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:19:56.723 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:19:56.726 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:19:56.727 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:19:56.727 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:57.681 INFO:teuthology.orchestra.run.smithi111.stdout:generic/242 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:19:57.693 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/242 2024-05-27T01:19:57.693 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/242 2024-05-27T01:19:57.693 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:19:57.697 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:58.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:19:58.114 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-05-27T01:19:58.443 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:19:58.449 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:19:58.449 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:19:58.450 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:19:58.450 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:58.939 INFO:teuthology.orchestra.run.smithi111.stdout:generic/243 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:19:58.948 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/243 2024-05-27T01:19:58.949 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/243 2024-05-27T01:19:58.949 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:19:58.951 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:59.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:19:59.286 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-05-27T01:19:59.463 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:19:59.467 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:19:59.467 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:19:59.468 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:19:59.468 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:59.681 INFO:teuthology.orchestra.run.smithi111.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:19:59.684 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/244 2024-05-27T01:19:59.684 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/244 2024-05-27T01:19:59.684 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:19:59.686 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:19:59.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:19:59.704 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-05-27T01:19:59.881 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:19:59.885 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:19:59.885 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:19:59.886 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:19:59.886 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:00.207 INFO:teuthology.orchestra.run.smithi111.stdout:generic/245 1s 2024-05-27T01:20:00.216 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/245 2024-05-27T01:20:00.216 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:00.218 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:00.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:00.267 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-05-27T01:20:00.681 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:00.685 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:00.686 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:00.686 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:00.687 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:00.979 INFO:teuthology.orchestra.run.smithi111.stdout:generic/246 0s 2024-05-27T01:20:01.008 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/246 2024-05-27T01:20:01.008 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:01.011 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:01.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:01.069 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-05-27T01:20:01.489 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:01.493 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:01.494 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:01.494 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:01.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:16.687 INFO:teuthology.orchestra.run.smithi111.stdout:generic/247 15s 2024-05-27T01:20:16.692 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/247 2024-05-27T01:20:16.692 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:16.695 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:16.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:16.704 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-05-27T01:20:16.885 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:16.889 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:16.889 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:16.890 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:16.890 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:17.226 INFO:teuthology.orchestra.run.smithi111.stdout:generic/248 1s 2024-05-27T01:20:17.232 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/248 2024-05-27T01:20:17.232 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:17.234 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:17.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:17.258 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-05-27T01:20:17.434 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:17.438 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:17.438 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:17.439 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:17.440 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:17.896 INFO:teuthology.orchestra.run.smithi111.stdout:generic/249 0s 2024-05-27T01:20:17.901 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/249 2024-05-27T01:20:17.901 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:17.904 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:17.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:17.961 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-05-27T01:20:18.404 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:18.408 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:18.408 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:18.409 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:18.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:18.690 INFO:teuthology.orchestra.run.smithi111.stdout:generic/250 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:20:18.693 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/250 2024-05-27T01:20:18.694 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/250 2024-05-27T01:20:18.694 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:18.696 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:18.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:18.712 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-05-27T01:20:18.893 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:18.896 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:18.897 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:18.899 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:18.899 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:19.185 INFO:teuthology.orchestra.run.smithi111.stdout:generic/251 [not run] FITRIM not supported on /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:19.194 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/251 2024-05-27T01:20:19.194 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/251 2024-05-27T01:20:19.194 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:19.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:19.209 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-05-27T01:20:19.395 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:19.399 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:19.400 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:19.401 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:19.401 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:19.625 INFO:teuthology.orchestra.run.smithi111.stdout:generic/252 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:20:19.627 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/252 2024-05-27T01:20:19.627 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/252 2024-05-27T01:20:19.627 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:19.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:19.643 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-05-27T01:20:19.818 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:19.822 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:19.823 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:19.824 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:19.824 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:20.197 INFO:teuthology.orchestra.run.smithi111.stdout:generic/253 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:20:20.204 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/253 2024-05-27T01:20:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/253 2024-05-27T01:20:20.205 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:20.208 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:20.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:20.565 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-05-27T01:20:20.755 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:20.759 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:20.760 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:20.761 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:20.761 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:21.124 INFO:teuthology.orchestra.run.smithi111.stdout:generic/254 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:20:21.129 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/254 2024-05-27T01:20:21.129 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/254 2024-05-27T01:20:21.129 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:21.133 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:21.489 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:21.490 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-05-27T01:20:21.677 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:21.682 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:21.682 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:21.683 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:21.683 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:22.004 INFO:teuthology.orchestra.run.smithi111.stdout:generic/255 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:20:22.009 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/255 2024-05-27T01:20:22.009 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/255 2024-05-27T01:20:22.009 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:22.011 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:22.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:22.060 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-05-27T01:20:22.495 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:22.501 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:22.501 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:22.501 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:22.501 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:23.220 INFO:teuthology.orchestra.run.smithi111.stdout:generic/256 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:20:23.224 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/256 2024-05-27T01:20:23.224 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/256 2024-05-27T01:20:23.224 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:23.226 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:23.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:23.246 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-05-27T01:20:23.430 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:23.434 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:23.434 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:23.435 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:23.435 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:24.311 INFO:teuthology.orchestra.run.smithi111.stdout:generic/257 1s 2024-05-27T01:20:24.316 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/257 2024-05-27T01:20:24.316 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:24.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:24.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:24.389 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-05-27T01:20:24.837 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:24.844 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:24.844 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:24.846 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:24.846 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:25.115 INFO:teuthology.orchestra.run.smithi111.stdout:generic/258 [not run] ceph does not support negative timestamps 2024-05-27T01:20:25.119 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/258 2024-05-27T01:20:25.119 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/258 2024-05-27T01:20:25.119 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:25.121 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:25.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:25.148 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-05-27T01:20:25.325 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:25.329 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:25.330 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:25.330 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:25.330 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:25.696 INFO:teuthology.orchestra.run.smithi111.stdout:generic/259 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:20:25.703 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/259 2024-05-27T01:20:25.703 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/259 2024-05-27T01:20:25.703 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:25.705 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:26.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:26.057 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-05-27T01:20:26.237 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:26.241 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:26.241 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:26.242 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:26.242 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:26.529 INFO:teuthology.orchestra.run.smithi111.stdout:generic/260 [not run] FITRIM not supported on /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:26.537 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/260 2024-05-27T01:20:26.537 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/260 2024-05-27T01:20:26.537 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:26.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:26.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:26.564 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-05-27T01:20:26.741 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:26.745 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:26.745 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:26.746 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:26.746 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:27.105 INFO:teuthology.orchestra.run.smithi111.stdout:generic/261 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:20:27.110 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/261 2024-05-27T01:20:27.110 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/261 2024-05-27T01:20:27.110 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:27.114 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:27.479 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:27.479 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-05-27T01:20:27.661 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:27.665 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:27.665 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:27.666 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:27.666 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:28.023 INFO:teuthology.orchestra.run.smithi111.stdout:generic/262 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:20:28.029 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/262 2024-05-27T01:20:28.029 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/262 2024-05-27T01:20:28.030 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:20:28.033 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:20:28.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:20:28.363 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-05-27T01:20:28.551 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:20:28.555 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:20:28.555 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:20:28.556 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:20:28.556 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:09.652 INFO:teuthology.orchestra.run.smithi111.stdout:generic/263 41s 2024-05-27T01:21:09.657 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/263 2024-05-27T01:21:09.657 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:09.659 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:09.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:09.705 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-05-27T01:21:10.150 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:10.154 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:10.154 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:10.155 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:10.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:10.524 INFO:teuthology.orchestra.run.smithi111.stdout:generic/264 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:10.531 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/264 2024-05-27T01:21:10.531 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/264 2024-05-27T01:21:10.531 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:10.535 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:10.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:10.897 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-05-27T01:21:11.079 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:11.083 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:11.084 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:11.499 INFO:teuthology.orchestra.run.smithi111.stdout:generic/265 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:11.504 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/265 2024-05-27T01:21:11.504 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/265 2024-05-27T01:21:11.504 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:11.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:11.844 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-05-27T01:21:12.032 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:12.036 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:12.037 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:12.038 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:12.435 INFO:teuthology.orchestra.run.smithi111.stdout:generic/266 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:12.443 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/266 2024-05-27T01:21:12.443 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/266 2024-05-27T01:21:12.443 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:12.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:12.808 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-05-27T01:21:12.988 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:12.992 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:12.992 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:12.993 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:12.993 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:13.399 INFO:teuthology.orchestra.run.smithi111.stdout:generic/267 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:13.406 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/267 2024-05-27T01:21:13.406 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/267 2024-05-27T01:21:13.406 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:13.408 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:13.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:13.761 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-05-27T01:21:13.954 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:13.958 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:13.959 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:13.960 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:13.960 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:14.340 INFO:teuthology.orchestra.run.smithi111.stdout:generic/268 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:14.347 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/268 2024-05-27T01:21:14.347 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/268 2024-05-27T01:21:14.347 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:14.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:14.702 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:14.702 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-05-27T01:21:14.879 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:14.883 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:14.884 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:14.885 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:14.885 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:15.113 INFO:teuthology.orchestra.run.smithi111.stdout:generic/269 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:21:15.117 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/269 2024-05-27T01:21:15.117 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/269 2024-05-27T01:21:15.117 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:15.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:15.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:15.144 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-05-27T01:21:15.321 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:15.325 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:15.325 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:15.326 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:15.326 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:15.565 INFO:teuthology.orchestra.run.smithi111.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:21:15.569 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/270 2024-05-27T01:21:15.569 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/270 2024-05-27T01:21:15.569 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:15.571 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:15.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:15.601 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-05-27T01:21:15.782 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:15.786 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:15.787 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:15.788 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:15.788 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:16.167 INFO:teuthology.orchestra.run.smithi111.stdout:generic/271 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:16.178 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/271 2024-05-27T01:21:16.178 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/271 2024-05-27T01:21:16.178 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:16.182 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:16.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:16.539 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-05-27T01:21:16.726 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:16.730 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:16.731 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:16.731 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:16.732 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:17.124 INFO:teuthology.orchestra.run.smithi111.stdout:generic/272 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:17.133 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/272 2024-05-27T01:21:17.133 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/272 2024-05-27T01:21:17.133 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:17.138 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:17.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:17.476 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-05-27T01:21:17.664 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:17.668 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:17.668 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:17.669 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:17.669 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:17.903 INFO:teuthology.orchestra.run.smithi111.stdout:generic/273 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:21:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/273 2024-05-27T01:21:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/273 2024-05-27T01:21:17.907 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:17.909 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:17.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:17.924 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-05-27T01:21:18.099 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:18.106 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:18.106 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:18.106 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:18.106 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:18.330 INFO:teuthology.orchestra.run.smithi111.stdout:generic/274 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T01:21:18.334 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/274 2024-05-27T01:21:18.335 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/274 2024-05-27T01:21:18.335 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:18.337 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:18.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:18.356 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-05-27T01:21:18.537 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:18.540 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:18.541 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:18.542 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:18.542 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:18.762 INFO:teuthology.orchestra.run.smithi111.stdout:generic/275 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:21:18.766 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/275 2024-05-27T01:21:18.766 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/275 2024-05-27T01:21:18.766 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:18.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:18.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:18.795 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-05-27T01:21:18.981 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:18.985 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:18.985 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:18.986 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:18.986 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:19.380 INFO:teuthology.orchestra.run.smithi111.stdout:generic/276 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:19.386 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/276 2024-05-27T01:21:19.386 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/276 2024-05-27T01:21:19.386 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:19.390 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:19.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:19.747 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-05-27T01:21:19.926 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:19.930 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:19.931 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:19.932 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:19.932 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:20.237 INFO:teuthology.orchestra.run.smithi111.stdout:generic/277 [not run] file system doesn't support chattr +A 2024-05-27T01:21:20.241 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/277 2024-05-27T01:21:20.241 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/277 2024-05-27T01:21:20.241 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:20.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:20.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:20.274 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-05-27T01:21:20.460 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:20.464 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:20.465 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:20.466 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:20.466 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:20.921 INFO:teuthology.orchestra.run.smithi111.stdout:generic/278 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:20.926 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/278 2024-05-27T01:21:20.926 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/278 2024-05-27T01:21:20.926 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:20.932 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:21.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:21.222 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-05-27T01:21:21.405 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:21.408 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:21.409 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:21.410 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:21.410 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:21.802 INFO:teuthology.orchestra.run.smithi111.stdout:generic/279 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:21.808 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/279 2024-05-27T01:21:21.808 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/279 2024-05-27T01:21:21.808 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:21.811 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:22.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:22.171 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-05-27T01:21:22.354 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:22.358 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:22.358 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:22.359 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:22.359 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:22.605 INFO:teuthology.orchestra.run.smithi111.stdout:generic/280 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:21:22.609 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/280 2024-05-27T01:21:22.609 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/280 2024-05-27T01:21:22.609 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:22.611 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:22.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:22.630 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-05-27T01:21:22.813 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:22.817 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:22.818 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:22.819 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:22.819 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:23.202 INFO:teuthology.orchestra.run.smithi111.stdout:generic/281 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:23.209 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/281 2024-05-27T01:21:23.209 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/281 2024-05-27T01:21:23.209 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:23.214 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:23.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:23.570 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-05-27T01:21:23.753 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:23.757 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:23.758 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:23.759 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:23.759 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:24.156 INFO:teuthology.orchestra.run.smithi111.stdout:generic/282 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:24.163 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/282 2024-05-27T01:21:24.163 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/282 2024-05-27T01:21:24.163 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:24.166 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:24.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:24.505 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-05-27T01:21:24.698 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:24.702 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:24.702 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:24.704 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:24.704 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:25.100 INFO:teuthology.orchestra.run.smithi111.stdout:generic/283 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:25.111 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/283 2024-05-27T01:21:25.111 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/283 2024-05-27T01:21:25.111 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:25.113 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:25.455 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:25.455 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-05-27T01:21:25.639 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:25.643 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:25.644 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:25.645 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:25.645 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:26.037 INFO:teuthology.orchestra.run.smithi111.stdout:generic/284 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:26.043 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/284 2024-05-27T01:21:26.044 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/284 2024-05-27T01:21:26.044 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:26.048 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:26.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:26.395 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-05-27T01:21:26.574 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:26.578 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:26.578 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:26.579 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:26.580 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:generic/285 1s 2024-05-27T01:21:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/285 2024-05-27T01:21:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:27.891 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:28.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:28.301 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-05-27T01:21:28.793 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:28.797 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:28.797 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:28.799 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:28.799 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:29.100 INFO:teuthology.orchestra.run.smithi111.stdout:generic/286 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:21:29.106 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/286 2024-05-27T01:21:29.106 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/286 2024-05-27T01:21:29.106 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:29.108 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:29.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:29.145 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-05-27T01:21:29.409 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:29.414 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:29.415 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:29.416 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:29.416 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:29.897 INFO:teuthology.orchestra.run.smithi111.stdout:generic/287 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:29.903 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/287 2024-05-27T01:21:29.903 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/287 2024-05-27T01:21:29.903 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:29.908 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:30.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:30.325 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-05-27T01:21:30.584 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:30.588 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:30.589 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:30.590 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:30.590 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:31.038 INFO:teuthology.orchestra.run.smithi111.stdout:generic/288 [not run] FITRIM not supported on /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:31.057 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/288 2024-05-27T01:21:31.057 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/288 2024-05-27T01:21:31.057 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:31.068 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:31.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:31.081 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-05-27T01:21:31.357 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:31.367 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:31.372 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:31.373 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:31.373 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:31.869 INFO:teuthology.orchestra.run.smithi111.stdout:generic/289 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:31.876 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/289 2024-05-27T01:21:31.876 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/289 2024-05-27T01:21:31.876 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:31.878 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:32.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:32.288 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-05-27T01:21:32.558 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:32.564 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:32.564 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:32.565 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:32.565 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:33.075 INFO:teuthology.orchestra.run.smithi111.stdout:generic/290 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:33.081 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/290 2024-05-27T01:21:33.081 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/290 2024-05-27T01:21:33.081 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:33.088 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:33.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:33.488 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-05-27T01:21:33.747 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:33.753 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:33.753 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:33.753 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:33.754 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:34.220 INFO:teuthology.orchestra.run.smithi111.stdout:generic/291 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:34.230 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/291 2024-05-27T01:21:34.230 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/291 2024-05-27T01:21:34.230 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:34.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:34.626 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:34.626 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-05-27T01:21:34.809 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:34.813 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:34.813 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:34.814 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:34.814 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:35.197 INFO:teuthology.orchestra.run.smithi111.stdout:generic/292 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:35.202 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/292 2024-05-27T01:21:35.202 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/292 2024-05-27T01:21:35.202 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:35.205 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:35.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:35.593 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-05-27T01:21:35.778 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:35.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:35.782 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:35.782 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:35.783 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:36.168 INFO:teuthology.orchestra.run.smithi111.stdout:generic/293 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:36.176 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/293 2024-05-27T01:21:36.176 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/293 2024-05-27T01:21:36.176 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:36.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:36.518 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:36.519 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-05-27T01:21:36.702 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:36.703 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:36.704 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:36.704 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:36.705 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:37.186 INFO:teuthology.orchestra.run.smithi111.stdout:generic/294 1s 2024-05-27T01:21:37.198 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/294 2024-05-27T01:21:37.198 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:37.202 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:37.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:37.215 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-05-27T01:21:37.598 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:37.602 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:37.603 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:37.604 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:37.604 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:37.988 INFO:teuthology.orchestra.run.smithi111.stdout:generic/295 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:37.993 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/295 2024-05-27T01:21:37.994 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/295 2024-05-27T01:21:37.994 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:37.998 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:38.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:38.387 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-05-27T01:21:38.565 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:38.569 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:38.569 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:38.570 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:38.570 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:38.949 INFO:teuthology.orchestra.run.smithi111.stdout:generic/296 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/296 2024-05-27T01:21:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/296 2024-05-27T01:21:38.961 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:38.961 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:39.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:39.322 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-05-27T01:21:39.502 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:39.506 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:39.506 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:39.507 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:39.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:39.884 INFO:teuthology.orchestra.run.smithi111.stdout:generic/297 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:39.891 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/297 2024-05-27T01:21:39.891 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/297 2024-05-27T01:21:39.891 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:39.893 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:40.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:40.284 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-05-27T01:21:40.466 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:40.469 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:40.470 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:40.471 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:40.471 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:40.858 INFO:teuthology.orchestra.run.smithi111.stdout:generic/298 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:40.863 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/298 2024-05-27T01:21:40.863 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/298 2024-05-27T01:21:40.863 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:40.867 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:41.226 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:41.227 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-05-27T01:21:41.413 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:41.417 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:41.417 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:41.418 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:41.418 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:41.711 INFO:teuthology.orchestra.run.smithi111.stdout:generic/299 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:21:41.715 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/299 2024-05-27T01:21:41.716 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/299 2024-05-27T01:21:41.716 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:41.718 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:41.738 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:41.739 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-05-27T01:21:41.922 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:41.926 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:41.926 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:41.928 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:41.928 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:42.194 INFO:teuthology.orchestra.run.smithi111.stdout:generic/300 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:21:42.198 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/300 2024-05-27T01:21:42.198 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/300 2024-05-27T01:21:42.198 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:42.200 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:42.220 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:42.220 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-05-27T01:21:42.405 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:42.409 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:42.409 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:42.410 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:42.410 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:42.782 INFO:teuthology.orchestra.run.smithi111.stdout:generic/301 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:42.788 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/301 2024-05-27T01:21:42.789 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/301 2024-05-27T01:21:42.789 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:42.792 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:43.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:43.127 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-05-27T01:21:43.320 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:43.323 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:43.324 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:43.325 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:43.325 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:43.707 INFO:teuthology.orchestra.run.smithi111.stdout:generic/302 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/302 2024-05-27T01:21:43.719 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/302 2024-05-27T01:21:43.719 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:43.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:44.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:44.027 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-05-27T01:21:44.223 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:44.227 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:44.228 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:44.228 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:44.229 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:44.483 INFO:teuthology.orchestra.run.smithi111.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:21:44.487 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/303 2024-05-27T01:21:44.487 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/303 2024-05-27T01:21:44.487 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:44.489 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:44.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:44.541 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-05-27T01:21:45.009 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:45.012 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:45.012 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:45.013 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:45.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:45.257 INFO:teuthology.orchestra.run.smithi111.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T01:21:45.261 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/304 2024-05-27T01:21:45.261 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/304 2024-05-27T01:21:45.261 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:45.264 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:45.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:45.322 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-05-27T01:21:45.756 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:45.760 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:45.760 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:45.761 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:45.761 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:46.123 INFO:teuthology.orchestra.run.smithi111.stdout:generic/305 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:21:46.130 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/305 2024-05-27T01:21:46.130 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/305 2024-05-27T01:21:46.130 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:46.134 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:46.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:46.462 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-05-27T01:21:46.652 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:46.656 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:46.656 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:46.657 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:46.658 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:47.180 INFO:teuthology.orchestra.run.smithi111.stdout:generic/306 1s 2024-05-27T01:21:47.187 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/306 2024-05-27T01:21:47.188 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:47.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:47.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:47.475 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-05-27T01:21:47.933 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:47.938 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:47.939 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:47.940 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:47.940 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:49.455 INFO:teuthology.orchestra.run.smithi111.stdout:generic/307 1s 2024-05-27T01:21:49.461 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/307 2024-05-27T01:21:49.461 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:49.463 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:49.473 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:49.473 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-05-27T01:21:49.671 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:49.675 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:49.675 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:49.676 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:49.676 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:49.927 INFO:teuthology.orchestra.run.smithi111.stdout:generic/308 0s 2024-05-27T01:21:49.932 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/308 2024-05-27T01:21:49.933 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:49.935 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:49.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:49.957 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-05-27T01:21:50.134 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:50.138 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:50.139 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:50.140 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:50.140 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:51.424 INFO:teuthology.orchestra.run.smithi111.stdout:generic/309 1s 2024-05-27T01:21:51.431 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/309 2024-05-27T01:21:51.431 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:21:51.432 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:21:51.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:21:51.502 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-05-27T01:21:51.903 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:21:51.907 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:21:51.907 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:21:51.908 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:21:51.908 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:07.218 INFO:teuthology.orchestra.run.smithi111.stdout:generic/310 76s 2024-05-27T01:23:07.224 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/310 2024-05-27T01:23:07.224 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:07.226 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:07.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:07.342 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-05-27T01:23:07.687 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:07.691 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:07.691 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:07.692 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:07.693 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:07.971 INFO:teuthology.orchestra.run.smithi111.stdout:generic/311 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:23:07.976 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/311 2024-05-27T01:23:07.976 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/311 2024-05-27T01:23:07.976 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:07.978 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:07.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:07.992 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-05-27T01:23:08.177 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:08.181 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:08.182 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:08.183 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:08.183 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:08.405 INFO:teuthology.orchestra.run.smithi111.stdout:generic/312 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:23:08.409 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/312 2024-05-27T01:23:08.409 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/312 2024-05-27T01:23:08.410 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:08.412 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:08.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:08.432 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-05-27T01:23:08.601 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:08.605 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:08.605 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:08.606 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:08.606 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:08.657 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:23:08.660 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:23:12.868 INFO:teuthology.orchestra.run.smithi111.stdout:generic/313 4s 2024-05-27T01:23:12.922 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/313 2024-05-27T01:23:12.923 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:12.926 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:12.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:12.940 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-05-27T01:23:13.127 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:13.130 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:13.131 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:13.132 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:13.132 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:15.064 INFO:teuthology.orchestra.run.smithi111.stdout:generic/314 2s 2024-05-27T01:23:15.072 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/314 2024-05-27T01:23:15.072 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:15.075 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:15.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:15.090 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-05-27T01:23:15.275 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:15.279 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:15.280 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:15.281 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:15.281 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:15.498 INFO:teuthology.orchestra.run.smithi111.stdout:generic/315 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T01:23:15.502 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/315 2024-05-27T01:23:15.502 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/315 2024-05-27T01:23:15.502 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:15.504 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:15.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:15.518 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-05-27T01:23:15.697 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:15.701 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:15.701 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:15.702 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:15.702 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:16.010 INFO:teuthology.orchestra.run.smithi111.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T01:23:16.014 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/316 2024-05-27T01:23:16.014 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/316 2024-05-27T01:23:16.014 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:16.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:16.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:16.057 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-05-27T01:23:16.410 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:16.413 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:16.413 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:16.414 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:16.414 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:17.172 INFO:teuthology.orchestra.run.smithi111.stdout:generic/317 1s 2024-05-27T01:23:17.178 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/317 2024-05-27T01:23:17.178 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:17.181 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:17.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:17.190 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-05-27T01:23:17.368 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:17.372 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:17.373 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:17.373 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:17.374 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:17.865 INFO:teuthology.orchestra.run.smithi111.stdout:generic/318 0s 2024-05-27T01:23:17.871 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/318 2024-05-27T01:23:17.871 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:17.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:17.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:17.882 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-05-27T01:23:18.068 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:18.072 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:18.072 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:18.073 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:18.073 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:18.396 INFO:teuthology.orchestra.run.smithi111.stdout:generic/319 0s 2024-05-27T01:23:18.401 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/319 2024-05-27T01:23:18.402 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:18.404 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:18.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:18.413 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-05-27T01:23:18.588 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:18.592 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:18.593 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:18.594 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:18.594 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:18.810 INFO:teuthology.orchestra.run.smithi111.stdout:generic/320 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:23:18.814 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/320 2024-05-27T01:23:18.814 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/320 2024-05-27T01:23:18.814 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:18.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:18.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:18.832 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-05-27T01:23:19.014 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:19.018 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:19.018 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:19.019 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:19.020 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:19.233 INFO:teuthology.orchestra.run.smithi111.stdout:generic/321 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:23:19.237 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/321 2024-05-27T01:23:19.237 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/321 2024-05-27T01:23:19.237 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:19.239 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:19.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:19.251 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-05-27T01:23:19.438 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:19.442 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:19.442 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:19.443 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:19.443 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:19.655 INFO:teuthology.orchestra.run.smithi111.stdout:generic/322 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:23:19.659 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/322 2024-05-27T01:23:19.659 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/322 2024-05-27T01:23:19.659 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:23:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:19.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:23:19.673 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-05-27T01:23:19.850 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:23:19.853 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:23:19.854 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:23:19.855 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:23:19.855 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:23:38.765 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:23:38.768 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:24:08.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:24:08.934 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:24:39.018 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:24:39.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:25:09.183 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:25:09.186 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:25:09.218 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T01:25:09.218+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 309858) UID: 0 2024-05-27T01:25:09.218 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T01:25:09.218+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 309858) UID: 0 2024-05-27T01:25:09.219 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T01:25:09.218+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 309858) UID: 0 2024-05-27T01:25:09.219 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T01:25:09.218+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 309858) UID: 0 2024-05-27T01:25:20.591 INFO:teuthology.orchestra.run.smithi111.stdout:generic/323 121s 2024-05-27T01:25:20.596 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/323 2024-05-27T01:25:20.596 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:20.598 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:20.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:20.638 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-05-27T01:25:21.035 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:21.042 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:21.043 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:21.045 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:21.045 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:21.262 INFO:teuthology.orchestra.run.smithi111.stdout:generic/324 [not run] defragmentation not supported for fstype "ceph" 2024-05-27T01:25:21.265 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/324 2024-05-27T01:25:21.266 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/324 2024-05-27T01:25:21.266 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:21.268 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:21.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:21.279 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-05-27T01:25:21.469 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:21.473 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:21.473 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:21.475 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:21.475 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:21.690 INFO:teuthology.orchestra.run.smithi111.stdout:generic/325 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:25:21.694 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/325 2024-05-27T01:25:21.694 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/325 2024-05-27T01:25:21.694 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:21.696 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:21.707 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:21.708 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-05-27T01:25:21.905 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:21.909 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:21.909 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:21.910 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:21.911 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:22.261 INFO:teuthology.orchestra.run.smithi111.stdout:generic/326 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:22.267 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/326 2024-05-27T01:25:22.267 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/326 2024-05-27T01:25:22.267 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:22.271 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:22.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:22.423 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-05-27T01:25:22.698 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:22.702 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:22.702 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:22.703 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:22.703 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:23.043 INFO:teuthology.orchestra.run.smithi111.stdout:generic/327 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:23.051 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/327 2024-05-27T01:25:23.051 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/327 2024-05-27T01:25:23.051 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:23.055 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:23.190 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:23.191 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-05-27T01:25:23.504 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:23.508 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:23.509 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:23.510 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:23.510 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:23.864 INFO:teuthology.orchestra.run.smithi111.stdout:generic/328 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:23.871 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/328 2024-05-27T01:25:23.871 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/328 2024-05-27T01:25:23.871 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:23.877 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:24.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:24.014 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-05-27T01:25:24.329 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:24.332 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:24.333 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:24.334 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:24.334 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:24.713 INFO:teuthology.orchestra.run.smithi111.stdout:generic/329 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:24.720 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/329 2024-05-27T01:25:24.720 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/329 2024-05-27T01:25:24.720 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:24.722 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:24.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:24.830 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-05-27T01:25:25.136 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:25.140 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:25.140 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:25.141 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:25.141 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:25.503 INFO:teuthology.orchestra.run.smithi111.stdout:generic/330 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:25.509 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/330 2024-05-27T01:25:25.509 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/330 2024-05-27T01:25:25.509 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:25.518 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:25.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:25.632 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-05-27T01:25:25.939 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:25.943 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:25.944 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:25.944 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:25.945 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:26.312 INFO:teuthology.orchestra.run.smithi111.stdout:generic/331 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:26.317 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/331 2024-05-27T01:25:26.317 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/331 2024-05-27T01:25:26.318 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:26.322 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:26.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:26.452 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-05-27T01:25:26.740 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:26.744 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:26.745 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:26.746 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:26.746 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:27.103 INFO:teuthology.orchestra.run.smithi111.stdout:generic/332 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:27.110 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/332 2024-05-27T01:25:27.110 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/332 2024-05-27T01:25:27.110 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:27.113 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:27.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:27.183 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-05-27T01:25:27.548 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:27.552 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:27.552 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:27.553 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:27.554 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:generic/333 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/333 2024-05-27T01:25:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/333 2024-05-27T01:25:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:27.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:28.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:28.069 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-05-27T01:25:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:28.370 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:28.705 INFO:teuthology.orchestra.run.smithi111.stdout:generic/334 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:25:28.710 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/334 2024-05-27T01:25:28.710 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/334 2024-05-27T01:25:28.711 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:28.716 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:28.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:28.831 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-05-27T01:25:29.164 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:29.168 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:29.169 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:29.170 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:29.170 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:29.388 INFO:teuthology.orchestra.run.smithi111.stdout:generic/335 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:25:29.392 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/335 2024-05-27T01:25:29.392 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/335 2024-05-27T01:25:29.392 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:29.394 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:29.405 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:29.405 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-05-27T01:25:29.594 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:29.598 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:29.598 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:29.599 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:29.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:29.844 INFO:teuthology.orchestra.run.smithi111.stdout:generic/336 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:25:29.848 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/336 2024-05-27T01:25:29.848 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/336 2024-05-27T01:25:29.848 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:29.850 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:29.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:29.865 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-05-27T01:25:30.053 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:30.057 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:30.058 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:30.059 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:30.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:30.381 INFO:teuthology.orchestra.run.smithi111.stdout:generic/337 0s 2024-05-27T01:25:30.386 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/337 2024-05-27T01:25:30.387 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:30.389 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:30.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:30.398 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-05-27T01:25:30.571 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:30.575 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:30.576 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:30.577 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:30.577 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:30.794 INFO:teuthology.orchestra.run.smithi111.stdout:generic/338 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:25:30.798 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/338 2024-05-27T01:25:30.798 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/338 2024-05-27T01:25:30.799 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:25:30.801 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:25:30.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:25:30.814 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-05-27T01:25:30.986 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:25:30.990 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:25:30.991 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:25:30.992 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:25:30.992 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:20.905 INFO:teuthology.orchestra.run.smithi111.stdout:generic/339 49s 2024-05-27T01:26:21.072 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/339 2024-05-27T01:26:21.072 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:21.075 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:21.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:21.085 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-05-27T01:26:21.386 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:21.395 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:21.395 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:21.400 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:21.401 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:24.159 INFO:teuthology.orchestra.run.smithi111.stdout:generic/340 3s 2024-05-27T01:26:24.166 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/340 2024-05-27T01:26:24.167 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:24.177 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:24.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:24.189 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-05-27T01:26:24.637 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:24.643 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:24.643 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:24.645 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:24.645 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:24.963 INFO:teuthology.orchestra.run.smithi111.stdout:generic/341 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:26:24.972 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/341 2024-05-27T01:26:24.972 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/341 2024-05-27T01:26:24.972 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:24.975 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:25.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:25.002 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-05-27T01:26:25.212 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:25.215 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:25.216 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:25.217 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:25.217 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:25.434 INFO:teuthology.orchestra.run.smithi111.stdout:generic/342 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:26:25.438 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/342 2024-05-27T01:26:25.438 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/342 2024-05-27T01:26:25.438 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:25.441 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:25.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:25.454 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-05-27T01:26:25.635 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:25.638 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:25.639 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:25.640 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:25.640 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:25.868 INFO:teuthology.orchestra.run.smithi111.stdout:generic/343 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:26:25.872 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/343 2024-05-27T01:26:25.872 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/343 2024-05-27T01:26:25.872 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:25.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:25.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:25.894 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-05-27T01:26:26.071 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:26.074 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:26.075 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:26.076 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:26.076 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:30.490 INFO:teuthology.orchestra.run.smithi111.stdout:generic/344 4s 2024-05-27T01:26:30.500 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/344 2024-05-27T01:26:30.500 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:30.503 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:30.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:30.528 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-05-27T01:26:30.900 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:30.904 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:30.904 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:30.905 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:36.099 INFO:teuthology.orchestra.run.smithi111.stdout:generic/345 6s 2024-05-27T01:26:36.112 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/345 2024-05-27T01:26:36.112 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:36.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:36.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:36.129 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-05-27T01:26:36.577 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:36.582 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:36.582 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:36.584 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:36.584 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:39.185 INFO:teuthology.orchestra.run.smithi111.stdout:generic/346 3s 2024-05-27T01:26:39.195 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/346 2024-05-27T01:26:39.196 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:39.199 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:39.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:39.212 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-05-27T01:26:39.637 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:39.646 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:39.647 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:39.648 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:39.648 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:40.068 INFO:teuthology.orchestra.run.smithi111.stdout:generic/347 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:26:40.083 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/347 2024-05-27T01:26:40.083 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/347 2024-05-27T01:26:40.083 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:40.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:40.107 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-05-27T01:26:40.284 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:40.288 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:40.289 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:40.289 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:40.290 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:40.529 INFO:teuthology.orchestra.run.smithi111.stdout:generic/348 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:26:40.533 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/348 2024-05-27T01:26:40.533 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/348 2024-05-27T01:26:40.533 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:40.535 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:40.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:40.554 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-05-27T01:26:40.729 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:40.733 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:40.733 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:40.734 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:40.735 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:41.149 INFO:teuthology.orchestra.run.smithi111.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-05-27T01:26:41.153 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/349 2024-05-27T01:26:41.153 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/349 2024-05-27T01:26:41.153 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:41.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:41.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:41.175 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-05-27T01:26:41.369 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:41.373 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:41.374 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:41.375 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:41.375 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:41.989 INFO:teuthology.orchestra.run.smithi111.stdout:generic/350 0s 2024-05-27T01:26:41.994 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/350 2024-05-27T01:26:41.994 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:41.996 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:42.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:42.005 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-05-27T01:26:42.188 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:42.192 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:42.193 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:42.194 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:42.194 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:42.433 INFO:teuthology.orchestra.run.smithi111.stdout:generic/351 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:26:42.437 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/351 2024-05-27T01:26:42.438 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/351 2024-05-27T01:26:42.438 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:42.440 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:42.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:42.458 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-05-27T01:26:42.638 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:42.641 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:42.642 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:42.643 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:43.010 INFO:teuthology.orchestra.run.smithi111.stdout:generic/352 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:26:43.018 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/352 2024-05-27T01:26:43.018 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/352 2024-05-27T01:26:43.018 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:43.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:43.213 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-05-27T01:26:43.486 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:43.490 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:43.491 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:43.492 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:43.492 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:43.862 INFO:teuthology.orchestra.run.smithi111.stdout:generic/353 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:26:43.869 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/353 2024-05-27T01:26:43.869 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/353 2024-05-27T01:26:43.869 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:43.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:44.083 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:44.083 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-05-27T01:26:44.362 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:44.366 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:44.366 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:44.367 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:44.368 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:48.699 INFO:teuthology.orchestra.run.smithi111.stdout:generic/354 4s 2024-05-27T01:26:48.716 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/354 2024-05-27T01:26:48.716 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:48.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:48.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:48.733 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-05-27T01:26:49.166 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:49.174 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:49.175 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:49.177 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:49.177 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:50.095 INFO:teuthology.orchestra.run.smithi111.stdout:generic/355 1s 2024-05-27T01:26:50.101 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/355 2024-05-27T01:26:50.101 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:50.104 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:50.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:50.149 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-05-27T01:26:50.548 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:50.552 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:50.553 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:50.554 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:50.554 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:50.943 INFO:teuthology.orchestra.run.smithi111.stdout:generic/356 [not run] swapfiles are not supported 2024-05-27T01:26:50.946 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/356 2024-05-27T01:26:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/356 2024-05-27T01:26:50.947 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:50.949 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:50.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:50.965 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-05-27T01:26:51.150 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:51.154 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:51.154 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:51.155 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:51.500 INFO:teuthology.orchestra.run.smithi111.stdout:generic/357 [not run] swapfiles are not supported 2024-05-27T01:26:51.504 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/357 2024-05-27T01:26:51.504 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/357 2024-05-27T01:26:51.504 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:51.506 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:51.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:51.523 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-05-27T01:26:51.709 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:51.712 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:51.713 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:51.714 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:51.714 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:52.089 INFO:teuthology.orchestra.run.smithi111.stdout:generic/358 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:26:52.095 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/358 2024-05-27T01:26:52.095 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/358 2024-05-27T01:26:52.095 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:52.098 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:52.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:52.317 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-05-27T01:26:52.557 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:52.561 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:52.561 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:52.562 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:52.562 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:52.969 INFO:teuthology.orchestra.run.smithi111.stdout:generic/359 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:26:53.011 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/359 2024-05-27T01:26:53.011 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/359 2024-05-27T01:26:53.012 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:53.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:53.122 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-05-27T01:26:53.406 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:53.410 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:53.410 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:53.411 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:53.411 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:53.656 INFO:teuthology.orchestra.run.smithi111.stdout:generic/360 0s 2024-05-27T01:26:53.661 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/360 2024-05-27T01:26:53.662 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:53.664 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:53.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:53.682 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-05-27T01:26:53.860 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:53.864 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:53.865 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:53.866 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:53.866 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:54.083 INFO:teuthology.orchestra.run.smithi111.stdout:generic/361 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:26:54.087 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/361 2024-05-27T01:26:54.087 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/361 2024-05-27T01:26:54.087 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:54.089 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:54.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:54.103 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-05-27T01:26:54.292 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:54.296 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:54.296 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:54.297 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:54.297 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:54.507 INFO:teuthology.orchestra.run.smithi111.stdout:generic/362 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:54.511 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/362 2024-05-27T01:26:54.511 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/362 2024-05-27T01:26:54.511 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:54.514 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:54.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:54.527 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-05-27T01:26:54.706 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:54.710 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:54.710 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:54.711 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:54.711 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:54.921 INFO:teuthology.orchestra.run.smithi111.stdout:generic/363 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:54.925 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/363 2024-05-27T01:26:54.925 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/363 2024-05-27T01:26:54.925 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:54.927 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:54.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:54.941 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-05-27T01:26:55.124 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:55.128 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:55.128 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:55.129 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:55.129 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:55.348 INFO:teuthology.orchestra.run.smithi111.stdout:generic/364 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:55.352 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/364 2024-05-27T01:26:55.353 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/364 2024-05-27T01:26:55.353 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:55.355 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:55.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:55.369 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-05-27T01:26:55.551 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:55.556 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:55.556 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:55.557 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:55.557 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:55.772 INFO:teuthology.orchestra.run.smithi111.stdout:generic/365 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:55.776 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/365 2024-05-27T01:26:55.776 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/365 2024-05-27T01:26:55.776 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:55.778 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:55.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:55.792 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-05-27T01:26:55.980 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:55.984 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:55.984 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:55.985 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:55.985 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:56.200 INFO:teuthology.orchestra.run.smithi111.stdout:generic/366 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:56.203 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/366 2024-05-27T01:26:56.204 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/366 2024-05-27T01:26:56.204 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:56.206 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:56.220 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:56.220 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-05-27T01:26:56.408 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:56.412 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:56.412 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:56.413 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:56.619 INFO:teuthology.orchestra.run.smithi111.stdout:generic/367 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:56.622 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/367 2024-05-27T01:26:56.623 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/367 2024-05-27T01:26:56.623 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:56.625 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:56.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:56.655 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-05-27T01:26:56.837 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:56.841 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:56.841 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:56.842 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:56.842 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:57.054 INFO:teuthology.orchestra.run.smithi111.stdout:generic/368 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:57.058 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/368 2024-05-27T01:26:57.058 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/368 2024-05-27T01:26:57.058 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:57.060 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:57.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:57.074 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-05-27T01:26:57.267 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:57.271 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:57.271 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:57.272 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:57.272 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:57.484 INFO:teuthology.orchestra.run.smithi111.stdout:generic/369 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:57.487 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/369 2024-05-27T01:26:57.488 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/369 2024-05-27T01:26:57.488 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:57.490 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:57.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:57.503 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-05-27T01:26:57.690 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:57.694 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:57.694 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:57.695 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:57.695 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:57.905 INFO:teuthology.orchestra.run.smithi111.stdout:generic/370 [not run] this test requires richacl support on $SCRATCH_DEV 2024-05-27T01:26:57.909 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/370 2024-05-27T01:26:57.910 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/370 2024-05-27T01:26:57.910 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:57.912 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:57.926 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:57.927 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-05-27T01:26:58.106 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:58.109 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:58.110 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:58.111 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:58.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:58.339 INFO:teuthology.orchestra.run.smithi111.stdout:generic/371 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:26:58.342 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/371 2024-05-27T01:26:58.343 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/371 2024-05-27T01:26:58.343 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:58.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:58.364 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-05-27T01:26:58.551 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:58.555 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:58.555 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:58.556 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:58.556 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:58.923 INFO:teuthology.orchestra.run.smithi111.stdout:generic/372 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:26:58.930 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/372 2024-05-27T01:26:58.930 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/372 2024-05-27T01:26:58.930 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:58.932 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:59.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:59.088 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-05-27T01:26:59.344 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:26:59.348 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:26:59.348 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:26:59.349 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:26:59.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:59.716 INFO:teuthology.orchestra.run.smithi111.stdout:generic/373 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:26:59.722 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/373 2024-05-27T01:26:59.722 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/373 2024-05-27T01:26:59.722 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:26:59.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:26:59.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:26:59.923 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-05-27T01:27:00.162 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:00.166 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:00.166 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:00.167 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:00.167 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:00.480 INFO:teuthology.orchestra.run.smithi111.stdout:generic/374 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-05-27T01:27:00.546 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/374 2024-05-27T01:27:00.546 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/374 2024-05-27T01:27:00.546 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:00.551 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:00.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:00.671 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-05-27T01:27:00.958 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:00.961 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:00.962 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:00.963 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:00.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:01.247 INFO:teuthology.orchestra.run.smithi111.stdout:generic/375 0s 2024-05-27T01:27:01.252 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/375 2024-05-27T01:27:01.253 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:01.255 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:01.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:01.300 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-05-27T01:27:01.687 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:01.691 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:01.691 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:01.692 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:01.692 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:01.921 INFO:teuthology.orchestra.run.smithi111.stdout:generic/376 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:27:01.925 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/376 2024-05-27T01:27:01.925 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/376 2024-05-27T01:27:01.925 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:01.927 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:01.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:01.941 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-05-27T01:27:02.118 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:02.122 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:02.122 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:02.123 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:02.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:02.506 INFO:teuthology.orchestra.run.smithi111.stdout:generic/377 0s 2024-05-27T01:27:02.513 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/377 2024-05-27T01:27:02.514 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:02.516 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:02.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:02.525 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-05-27T01:27:02.714 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:02.719 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:02.719 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:02.720 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:02.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:03.157 INFO:teuthology.orchestra.run.smithi111.stdout:generic/378 1s 2024-05-27T01:27:03.162 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/378 2024-05-27T01:27:03.162 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:03.165 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:03.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:03.318 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-05-27T01:27:03.628 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:03.631 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:03.632 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:03.633 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:03.861 INFO:teuthology.orchestra.run.smithi111.stdout:generic/379 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:03.865 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/379 2024-05-27T01:27:03.865 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/379 2024-05-27T01:27:03.866 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:03.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:03.892 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-05-27T01:27:04.076 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:04.080 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:04.080 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:04.081 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:04.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:04.293 INFO:teuthology.orchestra.run.smithi111.stdout:generic/380 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:04.297 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/380 2024-05-27T01:27:04.297 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/380 2024-05-27T01:27:04.297 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:04.299 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:04.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:04.332 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-05-27T01:27:04.510 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:04.513 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:04.514 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:04.515 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:04.515 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:04.726 INFO:teuthology.orchestra.run.smithi111.stdout:generic/381 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:04.730 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/381 2024-05-27T01:27:04.730 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/381 2024-05-27T01:27:04.730 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:04.732 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:04.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:04.774 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-05-27T01:27:04.952 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:04.955 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:04.956 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:04.957 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:04.957 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:05.163 INFO:teuthology.orchestra.run.smithi111.stdout:generic/382 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:05.166 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/382 2024-05-27T01:27:05.167 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/382 2024-05-27T01:27:05.167 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:05.169 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:05.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:05.183 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-05-27T01:27:05.360 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:05.364 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:05.365 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:05.366 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:05.366 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:05.579 INFO:teuthology.orchestra.run.smithi111.stdout:generic/383 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:05.582 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/383 2024-05-27T01:27:05.583 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/383 2024-05-27T01:27:05.583 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:05.585 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:05.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:05.599 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-05-27T01:27:05.778 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:05.782 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:05.782 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:05.784 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:05.784 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:06.002 INFO:teuthology.orchestra.run.smithi111.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:06.006 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/384 2024-05-27T01:27:06.006 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/384 2024-05-27T01:27:06.007 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:06.009 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:06.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:06.023 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-05-27T01:27:06.203 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:06.207 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:06.208 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:06.422 INFO:teuthology.orchestra.run.smithi111.stdout:generic/385 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:06.426 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/385 2024-05-27T01:27:06.426 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/385 2024-05-27T01:27:06.426 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:06.428 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:06.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:06.442 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-05-27T01:27:06.628 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:06.631 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:06.632 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:06.633 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:06.633 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:06.842 INFO:teuthology.orchestra.run.smithi111.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:06.846 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/386 2024-05-27T01:27:06.846 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/386 2024-05-27T01:27:06.846 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:06.848 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:06.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:06.862 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-05-27T01:27:07.044 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:07.048 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:07.048 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:07.049 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:07.049 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:07.421 INFO:teuthology.orchestra.run.smithi111.stdout:generic/387 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:27:07.429 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/387 2024-05-27T01:27:07.429 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/387 2024-05-27T01:27:07.429 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:07.431 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:07.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:07.617 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-05-27T01:27:07.864 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:07.868 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:07.868 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:07.869 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:07.869 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:08.089 INFO:teuthology.orchestra.run.smithi111.stdout:generic/388 [not run] require 172.21.15.111:6789:/scratch to be local device 2024-05-27T01:27:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/388 2024-05-27T01:27:08.093 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/388 2024-05-27T01:27:08.094 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:08.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:08.110 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-05-27T01:27:08.289 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:08.292 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:08.293 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:08.294 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:08.512 INFO:teuthology.orchestra.run.smithi111.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-05-27T01:27:08.516 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/389 2024-05-27T01:27:08.516 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/389 2024-05-27T01:27:08.516 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:08.518 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:08.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:08.532 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-05-27T01:27:08.716 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:08.720 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:08.720 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:08.722 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:08.961 INFO:teuthology.orchestra.run.smithi111.stdout:generic/390 [not run] ceph does not support freezing 2024-05-27T01:27:08.965 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/390 2024-05-27T01:27:08.965 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/390 2024-05-27T01:27:08.965 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:08.968 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:08.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:09.000 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-05-27T01:27:09.185 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:09.189 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:09.189 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:09.190 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:09.418 INFO:teuthology.orchestra.run.smithi111.stdout:generic/391 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:27:09.422 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/391 2024-05-27T01:27:09.422 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/391 2024-05-27T01:27:09.422 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:09.424 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:09.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:09.442 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-05-27T01:27:09.631 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:09.635 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:09.635 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:09.636 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:09.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:09.921 INFO:teuthology.orchestra.run.smithi111.stdout:generic/392 [not run] ceph does not support shutdown 2024-05-27T01:27:09.930 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/392 2024-05-27T01:27:09.930 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/392 2024-05-27T01:27:09.930 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:09.932 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:09.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:09.945 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-05-27T01:27:10.132 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:10.136 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:10.136 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:10.137 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:10.138 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:10.772 INFO:teuthology.orchestra.run.smithi111.stdout:generic/393 0s 2024-05-27T01:27:10.778 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/393 2024-05-27T01:27:10.778 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:10.780 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:10.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:10.789 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-05-27T01:27:10.972 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:10.976 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:10.977 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:11.230 INFO:teuthology.orchestra.run.smithi111.stdout:generic/394 0s 2024-05-27T01:27:11.236 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/394 2024-05-27T01:27:11.236 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:11.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:11.288 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-05-27T01:27:11.680 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:11.684 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:11.685 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:11.686 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:12.030 INFO:teuthology.orchestra.run.smithi111.stdout:generic/395 [not run] kernel does not support ceph encryption 2024-05-27T01:27:12.059 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/395 2024-05-27T01:27:12.059 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/395 2024-05-27T01:27:12.059 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:12.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:12.075 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-05-27T01:27:12.265 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:12.270 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:12.271 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:12.272 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:12.732 INFO:teuthology.orchestra.run.smithi111.stdout:generic/396 [not run] kernel does not support ceph encryption 2024-05-27T01:27:12.757 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/396 2024-05-27T01:27:12.757 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/396 2024-05-27T01:27:12.757 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:12.759 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:12.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:12.777 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-05-27T01:27:13.025 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:13.029 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:13.030 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:13.031 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:13.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:13.491 INFO:teuthology.orchestra.run.smithi111.stdout:generic/397 [not run] kernel does not support ceph encryption 2024-05-27T01:27:13.499 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/397 2024-05-27T01:27:13.500 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/397 2024-05-27T01:27:13.500 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:13.502 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:13.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:13.516 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-05-27T01:27:13.697 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:13.701 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:13.702 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:13.703 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:13.703 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:14.040 INFO:teuthology.orchestra.run.smithi111.stdout:generic/398 [not run] kernel does not support ceph encryption 2024-05-27T01:27:14.063 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/398 2024-05-27T01:27:14.063 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/398 2024-05-27T01:27:14.063 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:14.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:14.080 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-05-27T01:27:14.259 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:14.263 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:14.264 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:14.265 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:14.265 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:14.595 INFO:teuthology.orchestra.run.smithi111.stdout:generic/399 [not run] kernel does not support ceph encryption 2024-05-27T01:27:14.604 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/399 2024-05-27T01:27:14.604 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/399 2024-05-27T01:27:14.604 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:14.606 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:14.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:14.630 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-05-27T01:27:14.807 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:14.812 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:14.812 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:14.813 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:14.813 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:15.025 INFO:teuthology.orchestra.run.smithi111.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T01:27:15.029 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/400 2024-05-27T01:27:15.029 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/400 2024-05-27T01:27:15.029 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:15.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:15.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:15.056 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-05-27T01:27:15.239 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:15.243 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:15.243 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:15.244 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:15.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:15.657 INFO:teuthology.orchestra.run.smithi111.stdout:generic/401 0s 2024-05-27T01:27:15.667 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/401 2024-05-27T01:27:15.667 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:15.671 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:15.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:15.684 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-05-27T01:27:16.025 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:16.029 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:16.029 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:16.030 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:16.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:16.364 INFO:teuthology.orchestra.run.smithi111.stdout:generic/402 [not run] filesystem ceph timestamp bounds are unknown 2024-05-27T01:27:16.372 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/402 2024-05-27T01:27:16.373 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/402 2024-05-27T01:27:16.373 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:16.375 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:16.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:16.389 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-05-27T01:27:16.572 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:16.577 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:16.577 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:16.578 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:16.578 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:17.061 INFO:teuthology.orchestra.run.smithi111.stdout:generic/403 1s 2024-05-27T01:27:17.067 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/403 2024-05-27T01:27:17.067 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:17.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:17.078 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-05-27T01:27:17.258 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:17.262 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:17.263 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:17.264 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:17.264 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:17.494 INFO:teuthology.orchestra.run.smithi111.stdout:generic/404 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:27:17.498 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/404 2024-05-27T01:27:17.498 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/404 2024-05-27T01:27:17.499 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:17.501 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:17.518 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:17.518 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-05-27T01:27:17.704 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:17.708 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:17.708 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:17.710 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:17.710 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:17.975 INFO:teuthology.orchestra.run.smithi111.stdout:generic/405 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:27:17.979 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/405 2024-05-27T01:27:17.979 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/405 2024-05-27T01:27:17.979 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:17.981 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:17.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:17.996 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-05-27T01:27:18.173 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:18.177 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:18.177 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:18.178 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:18.178 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:19.227 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:27:19.230 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:27:19.718 INFO:teuthology.orchestra.run.smithi111.stdout:generic/406 1s 2024-05-27T01:27:19.724 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/406 2024-05-27T01:27:19.724 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:19.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:19.735 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-05-27T01:27:19.918 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:19.922 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:19.922 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:19.923 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:19.923 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:20.181 INFO:teuthology.orchestra.run.smithi111.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:27:20.185 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/407 2024-05-27T01:27:20.185 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/407 2024-05-27T01:27:20.185 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:20.187 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:20.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:20.257 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-05-27T01:27:20.666 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:20.671 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:20.671 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:20.672 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:20.672 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:20.920 INFO:teuthology.orchestra.run.smithi111.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T01:27:20.924 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/408 2024-05-27T01:27:20.925 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/408 2024-05-27T01:27:20.925 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:20.926 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:20.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:20.985 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-05-27T01:27:21.354 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:21.358 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:21.359 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:21.359 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:21.360 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:21.572 INFO:teuthology.orchestra.run.smithi111.stdout:generic/409 [not run] require 172.21.15.111:6789:/scratch to be local device 2024-05-27T01:27:21.576 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/409 2024-05-27T01:27:21.576 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/409 2024-05-27T01:27:21.577 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:21.579 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:21.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:21.592 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-05-27T01:27:21.776 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:21.780 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:21.780 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:21.781 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:21.781 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:22.015 INFO:teuthology.orchestra.run.smithi111.stdout:generic/410 [not run] require 172.21.15.111:6789:/scratch to be local device 2024-05-27T01:27:22.019 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/410 2024-05-27T01:27:22.019 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/410 2024-05-27T01:27:22.019 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:22.021 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:22.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:22.035 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-05-27T01:27:22.232 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:22.236 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:22.236 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:22.237 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:22.237 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:22.459 INFO:teuthology.orchestra.run.smithi111.stdout:generic/411 [not run] require 172.21.15.111:6789:/scratch to be local device 2024-05-27T01:27:22.463 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/411 2024-05-27T01:27:22.463 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/411 2024-05-27T01:27:22.464 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:22.466 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:22.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:22.481 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-05-27T01:27:22.656 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:22.660 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:22.661 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:22.662 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:22.662 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:23.110 INFO:teuthology.orchestra.run.smithi111.stdout:generic/412 1s 2024-05-27T01:27:23.116 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/412 2024-05-27T01:27:23.116 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:23.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:23.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:23.127 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-05-27T01:27:23.309 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:23.313 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:23.313 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:23.314 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:23.314 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:23.605 INFO:teuthology.orchestra.run.smithi111.stdout:generic/413 [not run] mount 172.21.15.111:6789:/scratch with dax failed 2024-05-27T01:27:23.610 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/413 2024-05-27T01:27:23.610 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/413 2024-05-27T01:27:23.610 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:23.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:23.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:23.626 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-05-27T01:27:23.808 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:23.812 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:23.813 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:23.814 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:23.814 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:24.181 INFO:teuthology.orchestra.run.smithi111.stdout:generic/414 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:27:24.188 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/414 2024-05-27T01:27:24.188 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/414 2024-05-27T01:27:24.188 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:24.190 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:24.405 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:24.406 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-05-27T01:27:24.628 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:24.632 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:24.632 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:24.633 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:24.633 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:25.021 INFO:teuthology.orchestra.run.smithi111.stdout:generic/415 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:27:25.027 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/415 2024-05-27T01:27:25.027 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/415 2024-05-27T01:27:25.028 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:25.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:25.220 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:25.220 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-05-27T01:27:25.463 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:25.467 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:25.468 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:25.469 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:25.469 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:25.687 INFO:teuthology.orchestra.run.smithi111.stdout:generic/416 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:27:25.691 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/416 2024-05-27T01:27:25.691 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/416 2024-05-27T01:27:25.691 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:25.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:25.707 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:25.707 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-05-27T01:27:25.886 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:25.890 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:25.890 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:25.891 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:25.891 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:26.176 INFO:teuthology.orchestra.run.smithi111.stdout:generic/417 [not run] ceph does not support shutdown 2024-05-27T01:27:26.185 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/417 2024-05-27T01:27:26.185 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/417 2024-05-27T01:27:26.185 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:26.187 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:26.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:26.201 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-05-27T01:27:26.393 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:26.396 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:26.397 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:26.398 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:26.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:26.652 INFO:teuthology.orchestra.run.smithi111.stdout:generic/418 [not run] require 172.21.15.111:6789:/test to be valid block disk 2024-05-27T01:27:26.656 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/418 2024-05-27T01:27:26.656 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/418 2024-05-27T01:27:26.656 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:26.658 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:26.676 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:26.676 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-05-27T01:27:26.857 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:26.861 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:26.861 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:26.862 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:27.188 INFO:teuthology.orchestra.run.smithi111.stdout:generic/419 [not run] kernel does not support ceph encryption 2024-05-27T01:27:27.197 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/419 2024-05-27T01:27:27.198 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/419 2024-05-27T01:27:27.198 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:27.200 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:27.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:27.213 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-05-27T01:27:27.394 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:27.402 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:27.402 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:27.402 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:27.402 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:27.758 INFO:teuthology.orchestra.run.smithi111.stdout:generic/420 0s 2024-05-27T01:27:27.764 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/420 2024-05-27T01:27:27.764 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:27.766 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:27.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:27.833 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-05-27T01:27:28.212 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:28.216 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:28.216 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:28.217 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:28.217 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:28.566 INFO:teuthology.orchestra.run.smithi111.stdout:generic/421 [not run] kernel does not support ceph encryption 2024-05-27T01:27:28.575 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/421 2024-05-27T01:27:28.577 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/421 2024-05-27T01:27:28.577 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:28.578 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:28.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:28.591 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-05-27T01:27:28.777 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:28.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:28.781 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:28.782 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:28.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:29.016 INFO:teuthology.orchestra.run.smithi111.stdout:generic/422 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T01:27:29.020 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/422 2024-05-27T01:27:29.020 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/422 2024-05-27T01:27:29.021 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:29.023 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:29.041 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:29.041 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-05-27T01:27:29.228 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:29.232 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:29.232 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:29.233 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:29.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:29.534 INFO:teuthology.orchestra.run.smithi111.stdout:generic/423 0s 2024-05-27T01:27:29.540 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/423 2024-05-27T01:27:29.540 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:29.542 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:29.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:29.590 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-05-27T01:27:29.996 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:30.000 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:30.000 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:30.001 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:30.001 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:30.239 INFO:teuthology.orchestra.run.smithi111.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-05-27T01:27:30.243 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/424 2024-05-27T01:27:30.244 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/424 2024-05-27T01:27:30.244 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:30.246 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:30.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:30.270 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-05-27T01:27:30.449 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:30.453 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:30.454 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:30.455 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:30.455 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:30.714 INFO:teuthology.orchestra.run.smithi111.stdout:generic/425 [not run] xfs_io fiemap -a failed (old kernel/wrong fs?) 2024-05-27T01:27:30.717 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/425 2024-05-27T01:27:30.718 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/425 2024-05-27T01:27:30.718 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:27:30.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:30.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:27:30.774 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-05-27T01:27:31.147 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:27:31.151 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:27:31.151 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:27:31.152 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:27:31.152 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:27:49.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:27:49.378 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:28:00.840 INFO:teuthology.orchestra.run.smithi111.stdout:generic/426 29s 2024-05-27T01:28:00.931 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/426 2024-05-27T01:28:00.931 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:00.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:01.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:01.108 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-05-27T01:28:02.156 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:02.160 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:02.160 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:02.161 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:02.161 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:02.520 INFO:teuthology.orchestra.run.smithi111.stdout:generic/427 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:28:02.524 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/427 2024-05-27T01:28:02.525 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/427 2024-05-27T01:28:02.525 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:02.528 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:02.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:02.543 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-05-27T01:28:02.718 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:02.722 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:02.723 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:02.724 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:02.724 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:02.982 INFO:teuthology.orchestra.run.smithi111.stdout:generic/428 0s 2024-05-27T01:28:02.990 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/428 2024-05-27T01:28:02.990 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:02.993 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:03.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:03.012 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-05-27T01:28:03.192 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:03.195 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:03.196 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:03.197 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:03.197 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:03.538 INFO:teuthology.orchestra.run.smithi111.stdout:generic/429 [not run] kernel does not support ceph encryption 2024-05-27T01:28:03.548 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/429 2024-05-27T01:28:03.548 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/429 2024-05-27T01:28:03.548 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:03.550 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:03.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:03.563 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-05-27T01:28:03.744 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:03.748 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:03.748 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:03.749 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:03.749 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:04.068 INFO:teuthology.orchestra.run.smithi111.stdout:generic/430 1s 2024-05-27T01:28:04.073 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/430 2024-05-27T01:28:04.073 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:04.076 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:04.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:04.138 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-05-27T01:28:04.528 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:04.532 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:04.532 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:04.533 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:04.533 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:04.826 INFO:teuthology.orchestra.run.smithi111.stdout:generic/431 0s 2024-05-27T01:28:04.832 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/431 2024-05-27T01:28:04.832 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:04.834 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:04.899 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:04.899 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-05-27T01:28:05.259 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:05.263 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:05.264 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:05.264 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:05.265 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:05.560 INFO:teuthology.orchestra.run.smithi111.stdout:generic/432 0s 2024-05-27T01:28:05.566 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/432 2024-05-27T01:28:05.566 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:05.568 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:05.631 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:05.631 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-05-27T01:28:06.004 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:06.008 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:06.009 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:06.010 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:06.010 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:06.289 INFO:teuthology.orchestra.run.smithi111.stdout:generic/433 0s 2024-05-27T01:28:06.294 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/433 2024-05-27T01:28:06.294 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:06.297 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:06.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:06.350 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-05-27T01:28:06.730 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:06.734 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:06.735 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:06.736 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:06.736 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:07.041 INFO:teuthology.orchestra.run.smithi111.stdout:generic/434 1s 2024-05-27T01:28:07.046 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/434 2024-05-27T01:28:07.046 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:07.049 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:07.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:07.104 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-05-27T01:28:07.482 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:07.485 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:07.486 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:07.487 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:07.487 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:07.833 INFO:teuthology.orchestra.run.smithi111.stdout:generic/435 [not run] kernel does not support ceph encryption 2024-05-27T01:28:07.857 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/435 2024-05-27T01:28:07.857 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/435 2024-05-27T01:28:07.857 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:07.860 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:07.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:07.873 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-05-27T01:28:08.066 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:08.071 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:08.072 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:08.560 INFO:teuthology.orchestra.run.smithi111.stdout:generic/436 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:28:08.564 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/436 2024-05-27T01:28:08.565 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/436 2024-05-27T01:28:08.565 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:08.567 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:08.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:08.601 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-05-27T01:28:08.962 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:08.966 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:08.967 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:08.968 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:08.968 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:09.858 INFO:teuthology.orchestra.run.smithi111.stdout:generic/437 0s 2024-05-27T01:28:09.864 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/437 2024-05-27T01:28:09.864 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:28:09.866 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:09.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:28:09.887 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-05-27T01:28:10.069 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:28:10.073 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:28:10.074 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:28:10.075 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:28:10.075 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:28:20.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:28:20.669 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:28:50.778 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:28:50.781 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:29:12.929 INFO:teuthology.orchestra.run.smithi111.stdout:generic/438 62s 2024-05-27T01:29:12.934 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/438 2024-05-27T01:29:12.934 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:12.936 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:12.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:12.955 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-05-27T01:29:13.133 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:13.137 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:13.137 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:13.138 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:13.138 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:14.001 INFO:teuthology.orchestra.run.smithi111.stdout:generic/439 0s 2024-05-27T01:29:14.006 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/439 2024-05-27T01:29:14.006 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:14.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:14.022 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-05-27T01:29:14.204 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:14.208 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:14.208 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:14.209 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:14.209 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:14.553 INFO:teuthology.orchestra.run.smithi111.stdout:generic/440 [not run] kernel does not support ceph encryption 2024-05-27T01:29:14.563 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/440 2024-05-27T01:29:14.563 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/440 2024-05-27T01:29:14.563 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:14.565 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:14.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:14.590 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-05-27T01:29:14.773 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:14.777 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:14.778 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:14.779 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:14.779 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:15.076 INFO:teuthology.orchestra.run.smithi111.stdout:generic/441 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:29:15.080 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/441 2024-05-27T01:29:15.080 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/441 2024-05-27T01:29:15.080 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:15.082 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:15.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:15.098 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-05-27T01:29:15.276 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:15.280 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:15.280 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:15.281 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:15.282 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:15.517 INFO:teuthology.orchestra.run.smithi111.stdout:generic/442 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:29:15.521 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/442 2024-05-27T01:29:15.521 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/442 2024-05-27T01:29:15.521 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:15.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:15.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:15.538 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-05-27T01:29:15.718 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:15.722 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:15.722 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:15.723 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:15.723 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:15.976 INFO:teuthology.orchestra.run.smithi111.stdout:generic/443 0s 2024-05-27T01:29:15.982 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/443 2024-05-27T01:29:15.982 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:15.984 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:16.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:16.005 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-05-27T01:29:16.182 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:16.185 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:16.186 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:16.187 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:16.187 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:16.501 INFO:teuthology.orchestra.run.smithi111.stdout:generic/444 0s 2024-05-27T01:29:16.507 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/444 2024-05-27T01:29:16.507 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:16.509 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:16.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:16.532 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-05-27T01:29:16.723 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:16.727 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:16.727 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:16.728 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:16.728 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:17.281 INFO:teuthology.orchestra.run.smithi111.stdout:generic/445 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:29:17.285 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/445 2024-05-27T01:29:17.285 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/445 2024-05-27T01:29:17.286 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:17.288 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:17.320 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:17.320 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-05-27T01:29:17.700 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:17.704 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:17.704 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:17.705 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:17.705 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:20.902 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:29:20.904 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:29:35.775 INFO:teuthology.orchestra.run.smithi111.stdout:generic/446 18s 2024-05-27T01:29:35.786 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/446 2024-05-27T01:29:35.786 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:35.792 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:35.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:35.811 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-05-27T01:29:36.149 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:36.153 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:36.153 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:36.154 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:36.154 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:36.517 INFO:teuthology.orchestra.run.smithi111.stdout:generic/447 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:29:36.523 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/447 2024-05-27T01:29:36.527 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/447 2024-05-27T01:29:36.527 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:36.530 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:36.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:36.825 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-05-27T01:29:37.009 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:37.012 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:37.013 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:37.014 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:37.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:38.014 INFO:teuthology.orchestra.run.smithi111.stdout:generic/448 0s 2024-05-27T01:29:38.102 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/448 2024-05-27T01:29:38.102 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:38.105 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:38.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:38.144 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-05-27T01:29:38.534 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:38.537 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:38.538 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:38.539 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:38.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:38.987 INFO:teuthology.orchestra.run.smithi111.stdout:generic/449 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T01:29:38.991 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/449 2024-05-27T01:29:38.991 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/449 2024-05-27T01:29:38.991 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:38.994 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:39.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:39.045 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-05-27T01:29:39.438 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:39.441 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:39.442 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:39.443 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:39.443 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:40.083 INFO:teuthology.orchestra.run.smithi111.stdout:generic/450 1s 2024-05-27T01:29:40.092 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/450 2024-05-27T01:29:40.092 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:29:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:40.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:29:40.145 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-05-27T01:29:40.514 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:29:40.518 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:29:40.518 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:29:40.519 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:29:40.519 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:29:50.986 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:29:50.989 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:30:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:generic/451 30s 2024-05-27T01:30:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/451 2024-05-27T01:30:10.892 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:10.894 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:10.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:10.949 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-05-27T01:30:11.339 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:11.343 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:11.343 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:11.345 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:11.720 INFO:teuthology.orchestra.run.smithi111.stdout:generic/452 0s 2024-05-27T01:30:11.726 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/452 2024-05-27T01:30:11.726 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:11.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:11.737 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-05-27T01:30:11.922 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:11.926 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:11.927 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:11.927 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:12.144 INFO:teuthology.orchestra.run.smithi111.stdout:generic/453 [not run] ceph does not allow unrestricted byte streams for names 2024-05-27T01:30:12.148 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/453 2024-05-27T01:30:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/453 2024-05-27T01:30:12.149 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:12.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:12.166 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-05-27T01:30:12.342 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:12.346 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:12.347 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:12.348 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:12.578 INFO:teuthology.orchestra.run.smithi111.stdout:generic/454 [not run] ceph does not allow unrestricted byte streams for names 2024-05-27T01:30:12.582 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/454 2024-05-27T01:30:12.582 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/454 2024-05-27T01:30:12.582 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:12.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:12.610 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-05-27T01:30:12.794 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:12.798 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:12.798 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:12.799 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:12.799 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:13.113 INFO:teuthology.orchestra.run.smithi111.stdout:generic/455 [not run] This test requires a valid $LOGWRITES_DEV 2024-05-27T01:30:13.117 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/455 2024-05-27T01:30:13.117 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/455 2024-05-27T01:30:13.117 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:13.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:13.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:13.135 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-05-27T01:30:13.314 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:13.318 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:13.318 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:13.319 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:13.319 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:13.563 INFO:teuthology.orchestra.run.smithi111.stdout:generic/456 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T01:30:13.567 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/456 2024-05-27T01:30:13.567 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/456 2024-05-27T01:30:13.567 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:13.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:13.589 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-05-27T01:30:13.769 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:13.773 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:13.773 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:13.774 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:13.774 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:14.221 INFO:teuthology.orchestra.run.smithi111.stdout:generic/457 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:30:14.228 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/457 2024-05-27T01:30:14.228 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/457 2024-05-27T01:30:14.228 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:14.230 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:14.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:14.445 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-05-27T01:30:14.634 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:14.638 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:14.638 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:14.639 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:14.639 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:15.031 INFO:teuthology.orchestra.run.smithi111.stdout:generic/458 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T01:30:15.038 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/458 2024-05-27T01:30:15.039 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/458 2024-05-27T01:30:15.039 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:15.044 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:15.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:15.286 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-05-27T01:30:15.483 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:15.487 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:15.488 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:15.488 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:15.489 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:15.900 INFO:teuthology.orchestra.run.smithi111.stdout:generic/459 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:30:15.906 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/459 2024-05-27T01:30:15.906 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/459 2024-05-27T01:30:15.907 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:15.907 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:15.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:15.924 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-05-27T01:30:16.106 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:16.109 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:16.110 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:16.111 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:16.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:17.899 INFO:teuthology.orchestra.run.smithi111.stdout:generic/460 1s 2024-05-27T01:30:17.904 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/460 2024-05-27T01:30:17.904 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:17.907 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:17.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:17.916 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-05-27T01:30:18.107 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:18.111 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:18.111 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:18.113 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:18.113 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:18.467 INFO:teuthology.orchestra.run.smithi111.stdout:generic/461 [not run] ceph does not support shutdown 2024-05-27T01:30:18.471 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/461 2024-05-27T01:30:18.471 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/461 2024-05-27T01:30:18.471 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:18.473 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:18.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:18.508 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-05-27T01:30:18.753 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:18.758 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:18.759 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:18.760 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:18.760 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:19.097 INFO:teuthology.orchestra.run.smithi111.stdout:generic/462 [not run] mount 172.21.15.111:6789:/scratch with dax failed 2024-05-27T01:30:19.101 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/462 2024-05-27T01:30:19.101 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/462 2024-05-27T01:30:19.101 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:19.103 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:19.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:19.131 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-05-27T01:30:19.316 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:19.320 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:19.320 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:19.321 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:19.321 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:19.597 INFO:teuthology.orchestra.run.smithi111.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph 2024-05-27T01:30:19.603 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/463 2024-05-27T01:30:19.604 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/463 2024-05-27T01:30:19.604 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:30:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:19.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:30:19.670 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-05-27T01:30:20.039 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:30:20.042 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:30:20.043 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:30:20.044 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:30:20.044 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:30:21.147 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:30:21.150 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:30:51.238 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:30:51.245 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:31:15.502 INFO:teuthology.orchestra.run.smithi111.stdout:generic/464 55s 2024-05-27T01:31:15.507 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/464 2024-05-27T01:31:15.507 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:15.510 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:15.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:15.520 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-05-27T01:31:15.698 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:15.702 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:15.703 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:15.704 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:15.704 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:19.862 INFO:teuthology.orchestra.run.smithi111.stdout:generic/465 3s 2024-05-27T01:31:19.867 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/465 2024-05-27T01:31:19.868 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:19.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:19.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:19.888 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-05-27T01:31:20.222 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:20.226 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:20.226 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:20.227 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:20.227 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:20.441 INFO:teuthology.orchestra.run.smithi111.stdout:generic/466 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:31:20.445 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/466 2024-05-27T01:31:20.445 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/466 2024-05-27T01:31:20.445 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:20.448 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:20.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:20.462 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-05-27T01:31:20.645 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:20.649 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:20.649 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:20.650 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:20.650 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:21.836 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:31:21.839 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:31:27.416 INFO:teuthology.orchestra.run.smithi111.stdout:generic/467 7s 2024-05-27T01:31:27.536 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/467 2024-05-27T01:31:27.536 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:27.538 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:27.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:27.780 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-05-27T01:31:28.746 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:28.750 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:28.750 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:28.751 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:28.751 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:29.121 INFO:teuthology.orchestra.run.smithi111.stdout:generic/468 [not run] ceph does not support shutdown 2024-05-27T01:31:29.130 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/468 2024-05-27T01:31:29.130 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/468 2024-05-27T01:31:29.130 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:29.132 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:29.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:29.146 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-05-27T01:31:29.326 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:29.330 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:29.330 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:29.331 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:29.331 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:29.569 INFO:teuthology.orchestra.run.smithi111.stdout:generic/469 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T01:31:29.573 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/469 2024-05-27T01:31:29.573 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/469 2024-05-27T01:31:29.573 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:29.576 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:29.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:29.595 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-05-27T01:31:29.776 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:29.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:29.781 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:29.782 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:29.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:30.064 INFO:teuthology.orchestra.run.smithi111.stdout:generic/470 [not run] This test requires a valid $LOGWRITES_DEV 2024-05-27T01:31:30.067 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/470 2024-05-27T01:31:30.068 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/470 2024-05-27T01:31:30.068 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:31:30.070 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:30.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:31:30.086 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-05-27T01:31:30.267 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:31:30.271 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:31:30.271 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:31:30.272 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:31:30.272 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:31:52.293 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:31:52.295 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:32:14.728 INFO:teuthology.orchestra.run.smithi111.stdout:generic/471 44s 2024-05-27T01:32:14.745 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/471 2024-05-27T01:32:14.745 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:32:14.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:14.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:32:14.823 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-05-27T01:32:15.290 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:32:15.302 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:32:15.303 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:32:15.304 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:32:15.304 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:15.962 INFO:teuthology.orchestra.run.smithi111.stdout:generic/472 [not run] swapfiles are not supported 2024-05-27T01:32:15.968 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/472 2024-05-27T01:32:15.968 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/472 2024-05-27T01:32:15.968 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:32:15.972 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:16.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:32:16.031 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-05-27T01:32:16.313 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:32:16.316 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:32:16.317 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:32:16.318 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:32:16.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:16.576 INFO:teuthology.orchestra.run.smithi111.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-05-27T01:32:16.580 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/473 2024-05-27T01:32:16.580 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/473 2024-05-27T01:32:16.580 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:32:16.582 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:16.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:32:16.599 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-05-27T01:32:16.777 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:32:16.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:32:16.781 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:32:16.782 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:32:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:17.070 INFO:teuthology.orchestra.run.smithi111.stdout:generic/474 [not run] ceph does not support shutdown 2024-05-27T01:32:17.077 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/474 2024-05-27T01:32:17.078 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/474 2024-05-27T01:32:17.078 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:32:17.080 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:17.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:32:17.093 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-05-27T01:32:17.273 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:32:17.277 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:32:17.278 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:32:17.279 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:32:17.279 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:17.532 INFO:teuthology.orchestra.run.smithi111.stdout:generic/475 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T01:32:17.536 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/475 2024-05-27T01:32:17.536 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/475 2024-05-27T01:32:17.536 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T01:32:17.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:17.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T01:32:17.552 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-05-27T01:32:17.732 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T01:32:17.736 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T01:32:17.736 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T01:32:17.737 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T01:32:17.737 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T01:32:22.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:32:22.589 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:32:22.658 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T01:32:22.658+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 572353) UID: 0 2024-05-27T01:32:22.659 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T01:32:22.658+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 572353) UID: 0 2024-05-27T01:38:09.417 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T01:38:09.416+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572663) UID: 0 2024-05-27T01:38:09.418 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T01:38:09.417+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572663) UID: 0 2024-05-27T01:38:09.419 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T01:38:09.417+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572663) UID: 0 2024-05-27T01:38:09.419 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T01:38:09.417+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572663) UID: 0 2024-05-27T01:56:06.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:56:06.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T01:56:06.567 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T01:56:06.567+0000 7f76dfbad640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50106) UID: 0 2024-05-27T01:56:06.567 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T01:56:06.567+0000 7f6764b08640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50106) UID: 0 2024-05-27T01:56:06.567 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T01:56:06.567+0000 7f2307fdb640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50106) UID: 0 2024-05-27T01:56:06.568 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T01:56:06.567+0000 7f62b9de8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50106) UID: 0 2024-05-27T01:56:07.223 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T01:56:07.223+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 572899) UID: 0 2024-05-27T01:56:07.224 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T01:56:07.224+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 572899) UID: 0 2024-05-27T02:05:29.250 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T02:05:29.249+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572965) UID: 0 2024-05-27T02:05:29.251 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T02:05:29.248+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572965) UID: 0 2024-05-27T02:05:29.251 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T02:05:29.251+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572965) UID: 0 2024-05-27T02:05:29.253 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T02:05:29.249+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 572965) UID: 0 2024-05-27T02:16:38.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T02:16:38.368 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T02:16:40.088 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T02:16:40.088+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 573096) UID: 0 2024-05-27T02:16:40.088 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T02:16:40.088+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 573096) UID: 0 2024-05-27T02:26:51.497 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T02:26:51.496+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 573192) UID: 0 2024-05-27T02:26:51.497 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T02:26:51.496+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 573192) UID: 0 2024-05-27T02:26:51.503 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T02:26:51.499+0000 7f2e7fb39640 -1 Fail to open '/proc/573192/cmdline' error = (2) No such file or directory 2024-05-27T02:26:51.503 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T02:26:51.499+0000 7f2e7fb39640 -1 received signal: Hangup from (PID: 573192) UID: 0 2024-05-27T02:26:51.503 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T02:26:51.502+0000 7f3531e31640 -1 Fail to open '/proc/573192/cmdline' error = (2) No such file or directory 2024-05-27T02:26:51.503 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T02:26:51.502+0000 7f3531e31640 -1 received signal: Hangup from (PID: 573192) UID: 0 2024-05-27T02:46:59.327 INFO:teuthology.orchestra.run.smithi111.stdout:generic/476 4482s 2024-05-27T02:46:59.505 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/476 2024-05-27T02:46:59.506 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:46:59.508 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:46:59.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:46:59.552 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-05-27T02:47:01.887 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:47:01.892 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:47:01.892 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:47:01.893 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:47:01.893 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:49:26.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T02:49:26.411 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T02:49:26.465 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T02:49:26.465+0000 7f62b9de8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50677) UID: 0 2024-05-27T02:49:26.465 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T02:49:26.465+0000 7f2307fdb640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50677) UID: 0 2024-05-27T02:49:26.465 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T02:49:26.465+0000 7f76dfbad640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50677) UID: 0 2024-05-27T02:49:26.466 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T02:49:26.465+0000 7f6764b08640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50677) UID: 0 2024-05-27T02:49:26.472 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T02:49:26.472+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 573763) UID: 0 2024-05-27T02:49:26.472 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T02:49:26.472+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 573763) UID: 0 2024-05-27T02:53:54.494 INFO:teuthology.orchestra.run.smithi111.stdout:generic/477 29s 2024-05-27T02:53:54.638 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/477 2024-05-27T02:53:54.638 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:53:54.641 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:53:54.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:53:54.832 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-05-27T02:53:57.479 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:53:57.484 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:53:57.484 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:53:57.485 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:53:57.485 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:08.751 INFO:teuthology.orchestra.run.smithi111.stdout:generic/478 11s 2024-05-27T02:54:08.764 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/478 2024-05-27T02:54:08.764 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:08.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:08.796 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-05-27T02:54:09.074 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:09.079 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:09.080 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:09.081 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:09.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:09.718 INFO:teuthology.orchestra.run.smithi111.stdout:generic/479 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:09.724 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/479 2024-05-27T02:54:09.724 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/479 2024-05-27T02:54:09.724 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:09.728 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:09.760 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:09.760 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-05-27T02:54:10.018 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:10.024 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:10.025 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:10.027 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:10.027 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:10.385 INFO:teuthology.orchestra.run.smithi111.stdout:generic/480 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:10.394 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/480 2024-05-27T02:54:10.394 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/480 2024-05-27T02:54:10.394 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:10.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:10.424 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:10.424 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-05-27T02:54:10.676 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:10.680 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:10.681 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:10.682 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:11.014 INFO:teuthology.orchestra.run.smithi111.stdout:generic/481 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:11.018 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/481 2024-05-27T02:54:11.018 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/481 2024-05-27T02:54:11.019 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:11.050 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:11.050 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-05-27T02:54:11.322 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:11.326 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:11.326 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:11.327 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:11.710 INFO:teuthology.orchestra.run.smithi111.stdout:generic/482 [not run] This test requires a valid $LOGWRITES_DEV 2024-05-27T02:54:11.714 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/482 2024-05-27T02:54:11.714 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/482 2024-05-27T02:54:11.714 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:11.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:11.734 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-05-27T02:54:11.979 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:11.987 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:11.988 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:11.990 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:12.312 INFO:teuthology.orchestra.run.smithi111.stdout:generic/483 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:12.318 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/483 2024-05-27T02:54:12.318 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/483 2024-05-27T02:54:12.318 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:12.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:12.349 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-05-27T02:54:12.596 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:12.601 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:12.601 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:12.602 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:12.954 INFO:teuthology.orchestra.run.smithi111.stdout:generic/484 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:12.959 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/484 2024-05-27T02:54:12.959 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/484 2024-05-27T02:54:12.959 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:12.962 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:12.983 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:12.983 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-05-27T02:54:13.254 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:13.260 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:13.261 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:13.262 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:13.262 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:13.589 INFO:teuthology.orchestra.run.smithi111.stdout:generic/485 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T02:54:13.596 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/485 2024-05-27T02:54:13.596 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/485 2024-05-27T02:54:13.596 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:13.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:13.623 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-05-27T02:54:13.874 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:13.879 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:13.879 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:13.881 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:13.881 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:14.639 INFO:teuthology.orchestra.run.smithi111.stdout:generic/486 0s 2024-05-27T02:54:14.646 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/486 2024-05-27T02:54:14.646 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:14.649 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:14.660 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:14.660 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-05-27T02:54:14.914 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:14.919 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:14.919 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:14.921 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:14.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:15.252 INFO:teuthology.orchestra.run.smithi111.stdout:generic/487 [not run] This test requires a valid $SCRATCH_LOGDEV 2024-05-27T02:54:15.257 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/487 2024-05-27T02:54:15.258 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/487 2024-05-27T02:54:15.258 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:15.261 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:15.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:15.288 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-05-27T02:54:15.553 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:15.559 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:15.559 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:15.560 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:15.560 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:15.892 INFO:teuthology.orchestra.run.smithi111.stdout:generic/488 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-05-27T02:54:15.897 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/488 2024-05-27T02:54:15.897 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/488 2024-05-27T02:54:15.897 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:15.900 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:15.928 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:15.928 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-05-27T02:54:16.203 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:16.208 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:16.209 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:16.210 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:16.210 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:16.528 INFO:teuthology.orchestra.run.smithi111.stdout:generic/489 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:16.535 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/489 2024-05-27T02:54:16.535 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/489 2024-05-27T02:54:16.535 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:16.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:16.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:16.569 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-05-27T02:54:16.816 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:16.825 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:16.825 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:16.828 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:16.828 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:18.762 INFO:teuthology.orchestra.run.smithi111.stdout:generic/490 2s 2024-05-27T02:54:18.772 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/490 2024-05-27T02:54:18.772 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:18.775 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:18.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:18.871 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-05-27T02:54:19.352 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:19.358 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:19.359 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:19.360 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:19.360 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:19.976 INFO:teuthology.orchestra.run.smithi111.stdout:generic/491 [not run] ceph does not support freezing 2024-05-27T02:54:19.980 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/491 2024-05-27T02:54:19.980 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/491 2024-05-27T02:54:19.980 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:19.984 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:20.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:20.002 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-05-27T02:54:20.330 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:20.335 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:20.336 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:20.339 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:20.339 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:20.781 INFO:teuthology.orchestra.run.smithi111.stdout:generic/492 [not run] xfs_io label failed (old kernel/wrong fs?) 2024-05-27T02:54:20.790 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/492 2024-05-27T02:54:20.790 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/492 2024-05-27T02:54:20.790 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:20.790 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:20.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:20.810 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-05-27T02:54:21.063 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:21.068 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:21.069 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:21.070 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:21.070 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:21.778 INFO:teuthology.orchestra.run.smithi111.stdout:generic/493 [not run] swapfiles are not supported 2024-05-27T02:54:21.785 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/493 2024-05-27T02:54:21.785 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/493 2024-05-27T02:54:21.785 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:21.788 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:21.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:21.815 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-05-27T02:54:22.084 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:22.089 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:22.090 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:22.091 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:22.092 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:22.579 INFO:teuthology.orchestra.run.smithi111.stdout:generic/494 [not run] swapfiles are not supported 2024-05-27T02:54:22.584 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/494 2024-05-27T02:54:22.584 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/494 2024-05-27T02:54:22.584 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:22.588 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:22.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:22.624 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-05-27T02:54:22.890 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:22.896 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:22.896 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:22.897 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:22.899 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:23.424 INFO:teuthology.orchestra.run.smithi111.stdout:generic/495 [not run] swapfiles are not supported 2024-05-27T02:54:23.429 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/495 2024-05-27T02:54:23.429 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/495 2024-05-27T02:54:23.429 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:23.436 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:23.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:23.460 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-05-27T02:54:23.724 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:23.729 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:23.730 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:23.731 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:23.731 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:24.270 INFO:teuthology.orchestra.run.smithi111.stdout:generic/496 [not run] swapfiles are not supported 2024-05-27T02:54:24.275 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/496 2024-05-27T02:54:24.275 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/496 2024-05-27T02:54:24.275 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:24.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:24.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:24.301 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-05-27T02:54:24.563 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:24.568 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:24.568 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:24.569 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:24.569 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:25.086 INFO:teuthology.orchestra.run.smithi111.stdout:generic/497 [not run] swapfiles are not supported 2024-05-27T02:54:25.091 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/497 2024-05-27T02:54:25.091 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/497 2024-05-27T02:54:25.091 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:25.094 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:25.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:25.125 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-05-27T02:54:25.426 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:25.432 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:25.433 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:25.435 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:25.435 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:25.778 INFO:teuthology.orchestra.run.smithi111.stdout:generic/498 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:25.783 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/498 2024-05-27T02:54:25.783 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/498 2024-05-27T02:54:25.783 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:25.787 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:25.816 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:25.817 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-05-27T02:54:26.056 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:26.060 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:26.062 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:26.062 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:26.062 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:26.365 INFO:teuthology.orchestra.run.smithi111.stdout:generic/499 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T02:54:26.370 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/499 2024-05-27T02:54:26.370 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/499 2024-05-27T02:54:26.370 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:26.373 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:26.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:26.402 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-05-27T02:54:26.648 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:26.654 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:26.655 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:26.656 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:26.656 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:27.026 INFO:teuthology.orchestra.run.smithi111.stdout:generic/500 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:27.031 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/500 2024-05-27T02:54:27.031 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/500 2024-05-27T02:54:27.031 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:27.034 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:27.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:27.056 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-05-27T02:54:27.312 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:27.317 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:27.317 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:27.319 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:27.319 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:27.858 INFO:teuthology.orchestra.run.smithi111.stdout:generic/501 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T02:54:27.864 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/501 2024-05-27T02:54:27.864 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/501 2024-05-27T02:54:27.865 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:27.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:28.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:28.151 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-05-27T02:54:28.411 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:28.415 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:28.415 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:28.416 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:28.416 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:28.756 INFO:teuthology.orchestra.run.smithi111.stdout:generic/502 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:28.761 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/502 2024-05-27T02:54:28.761 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/502 2024-05-27T02:54:28.761 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:28.764 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:28.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:28.803 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-05-27T02:54:29.029 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:29.034 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:29.034 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:29.035 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:29.035 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:29.361 INFO:teuthology.orchestra.run.smithi111.stdout:generic/503 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T02:54:29.367 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/503 2024-05-27T02:54:29.367 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/503 2024-05-27T02:54:29.367 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:29.370 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:29.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:29.400 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-05-27T02:54:29.635 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:29.641 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:29.641 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:29.643 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:29.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:29.978 INFO:teuthology.orchestra.run.smithi111.stdout:generic/504 0s 2024-05-27T02:54:29.984 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/504 2024-05-27T02:54:29.984 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:29.987 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:30.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:30.027 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-05-27T02:54:30.274 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:30.279 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:30.280 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:30.281 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:30.281 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:30.707 INFO:teuthology.orchestra.run.smithi111.stdout:generic/505 [not run] ceph does not support shutdown 2024-05-27T02:54:30.735 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/505 2024-05-27T02:54:30.735 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/505 2024-05-27T02:54:30.735 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:30.737 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:30.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:30.757 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-05-27T02:54:31.026 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:31.036 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:31.037 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:31.038 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:31.038 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:31.341 INFO:teuthology.orchestra.run.smithi111.stdout:generic/506 [not run] disk quotas not supported by this filesystem type: ceph 2024-05-27T02:54:31.348 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/506 2024-05-27T02:54:31.348 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/506 2024-05-27T02:54:31.348 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:31.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:31.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:31.371 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-05-27T02:54:31.607 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:31.613 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:31.613 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:31.614 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:31.614 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:31.985 INFO:teuthology.orchestra.run.smithi111.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-05-27T02:54:31.990 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/507 2024-05-27T02:54:31.991 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/507 2024-05-27T02:54:31.991 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:31.993 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:32.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:32.029 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-05-27T02:54:32.279 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:32.283 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:32.284 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:32.285 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:32.285 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:32.572 INFO:teuthology.orchestra.run.smithi111.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph 2024-05-27T02:54:32.577 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/508 2024-05-27T02:54:32.577 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/508 2024-05-27T02:54:32.577 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:32.580 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:32.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:32.599 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-05-27T02:54:32.842 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:32.847 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:32.847 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:32.848 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:32.848 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:33.162 INFO:teuthology.orchestra.run.smithi111.stdout:generic/509 [not run] O_TMPFILE is not supported 2024-05-27T02:54:33.168 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/509 2024-05-27T02:54:33.168 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/509 2024-05-27T02:54:33.168 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:33.171 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:33.190 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:33.190 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-05-27T02:54:33.427 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:33.434 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:33.434 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:33.437 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:33.437 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:33.778 INFO:teuthology.orchestra.run.smithi111.stdout:generic/510 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:33.788 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/510 2024-05-27T02:54:33.788 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/510 2024-05-27T02:54:33.788 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:33.791 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:33.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:33.812 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-05-27T02:54:34.076 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:34.082 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:34.083 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:34.084 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:34.084 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:34.449 INFO:teuthology.orchestra.run.smithi111.stdout:generic/511 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-05-27T02:54:34.454 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/511 2024-05-27T02:54:34.454 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/511 2024-05-27T02:54:34.454 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:34.457 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:34.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:34.486 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-05-27T02:54:34.751 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:34.756 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:34.757 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:34.758 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:34.758 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:35.133 INFO:teuthology.orchestra.run.smithi111.stdout:generic/512 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-05-27T02:54:35.138 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/512 2024-05-27T02:54:35.138 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/512 2024-05-27T02:54:35.139 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:35.142 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:35.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:35.175 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-05-27T02:54:35.468 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:35.474 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:35.474 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:35.476 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:35.476 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:36.041 INFO:teuthology.orchestra.run.smithi111.stdout:generic/513 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T02:54:36.046 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/513 2024-05-27T02:54:36.047 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/513 2024-05-27T02:54:36.047 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:36.050 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:36.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:36.300 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-05-27T02:54:36.586 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:36.591 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:36.591 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:36.593 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:36.593 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:37.188 INFO:teuthology.orchestra.run.smithi111.stdout:generic/514 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T02:54:37.193 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/514 2024-05-27T02:54:37.193 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/514 2024-05-27T02:54:37.193 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:37.197 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:37.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:37.432 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-05-27T02:54:37.757 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:37.762 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:37.762 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:37.764 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:37.764 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:38.305 INFO:teuthology.orchestra.run.smithi111.stdout:generic/515 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T02:54:38.311 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/515 2024-05-27T02:54:38.311 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/515 2024-05-27T02:54:38.311 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:38.314 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:38.585 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:38.585 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-05-27T02:54:38.905 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:38.910 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:38.910 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:38.911 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:38.911 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:39.254 INFO:teuthology.orchestra.run.smithi111.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph 2024-05-27T02:54:39.259 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/516 2024-05-27T02:54:39.259 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/516 2024-05-27T02:54:39.260 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:39.263 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:39.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:39.337 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-05-27T02:54:39.779 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:39.784 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:39.784 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:39.785 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:39.785 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:40.341 INFO:teuthology.orchestra.run.smithi111.stdout:generic/517 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-05-27T02:54:40.396 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/517 2024-05-27T02:54:40.396 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/517 2024-05-27T02:54:40.396 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:40.399 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:40.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:40.619 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-05-27T02:54:40.924 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:40.928 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:40.928 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:40.929 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:40.930 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:41.452 INFO:teuthology.orchestra.run.smithi111.stdout:generic/518 [not run] Reflink not supported by scratch filesystem type: ceph 2024-05-27T02:54:41.458 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/518 2024-05-27T02:54:41.458 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/518 2024-05-27T02:54:41.458 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:41.461 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:41.721 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:41.721 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-05-27T02:54:42.030 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:42.035 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:42.036 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:42.037 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:42.037 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:42.387 INFO:teuthology.orchestra.run.smithi111.stdout:generic/519 [not run] FIBMAP not supported by this filesystem 2024-05-27T02:54:42.391 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/519 2024-05-27T02:54:42.391 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/519 2024-05-27T02:54:42.391 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:42.397 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:42.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:42.463 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-05-27T02:54:42.963 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:42.968 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:42.969 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:42.970 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:43.281 INFO:teuthology.orchestra.run.smithi111.stdout:generic/520 [not run] require 172.21.15.111:6789:/scratch to be valid block disk 2024-05-27T02:54:43.285 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/520 2024-05-27T02:54:43.286 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/520 2024-05-27T02:54:43.286 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2024-05-27T02:54:43.288 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T02:54:43.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-05-27T02:54:43.313 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.pEH4Qq3ph1xfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-05-27T02:54:43.569 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph 2024-05-27T02:54:43.574 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 5.14.0-447.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 7 10:46:15 UTC 2024 2024-05-27T02:54:43.575 INFO:teuthology.orchestra.run.smithi111.stdout:MKFS_OPTIONS -- 172.21.15.111:6789:/scratch 2024-05-27T02:54:43.576 INFO:teuthology.orchestra.run.smithi111.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQD51VNmjllaHhAA6n5Xlidu+13A3cblyM3HFg== -o context=system_u:object_r:root_t:s0 172.21.15.111:6789:/scratch /tmp/tmp.dDEeBnibCbscratch 2024-05-27T02:54:43.576 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T03:06:30.550 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T03:06:30.548+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595330) UID: 0 2024-05-27T03:06:30.552 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:06:30.548+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595330) UID: 0 2024-05-27T03:06:30.552 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T03:06:30.548+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595330) UID: 0 2024-05-27T03:06:30.553 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T03:06:30.548+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595330) UID: 0 2024-05-27T03:21:38.275 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:21:38.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:21:38.316 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T03:21:38.316+0000 7f6764b08640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50734) UID: 0 2024-05-27T03:21:38.317 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T03:21:38.316+0000 7f76dfbad640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50734) UID: 0 2024-05-27T03:21:38.317 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T03:21:38.316+0000 7f2307fdb640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50734) UID: 0 2024-05-27T03:21:38.317 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T03:21:38.316+0000 7f62b9de8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50734) UID: 0 2024-05-27T03:21:38.371 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T03:21:38.371+0000 7f519a50e640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 595379) UID: 0 2024-05-27T03:21:38.372 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T03:21:38.371+0000 7faf37c21640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 595379) UID: 0 2024-05-27T03:25:52.677 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:25:52.675+0000 7fcaa45a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595384) UID: 0 2024-05-27T03:25:52.677 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T03:25:52.675+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595384) UID: 0 2024-05-27T03:25:52.678 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T03:25:52.675+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595384) UID: 0 2024-05-27T03:25:52.678 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T03:25:52.675+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595384) UID: 0 2024-05-27T03:29:00.372 INFO:tasks.ceph.osd.5.smithi153.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f76d59a5640 time 2024-05-27T03:29:00.370829+0000 2024-05-27T03:29:00.372 INFO:tasks.ceph.osd.5.smithi153.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-05-27T03:29:00.372 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.372 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x563c19dac176] 2024-05-27T03:29:00.372 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.372 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.373 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.373 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.373 INFO:tasks.ceph.osd.5.smithi153.stderr:*** Caught signal (Aborted) ** 2024-05-27T03:29:00.373 INFO:tasks.ceph.osd.5.smithi153.stderr: in thread 7f76d59a5640 thread_name:bstore_aio 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76e303e6f0] 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f76e308b94c] 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: raise() 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: abort() 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x563c19dac237] 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 7: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 8: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.374 INFO:tasks.ceph.osd.5.smithi153.stderr: 9: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T03:29:00.369+0000 7f76d59a5640 -1 bdev(0x563c1cb3d180 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T03:29:00.370+0000 7f76d59a5640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f76d59a5640 time 2024-05-27T03:29:00.370829+0000 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x563c19dac176] 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.375 INFO:tasks.ceph.osd.5.smithi153.stderr:2024-05-27T03:29:00.372+0000 7f76d59a5640 -1 *** Caught signal (Aborted) ** 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: in thread 7f76d59a5640 thread_name:bstore_aio 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76e303e6f0] 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f76e308b94c] 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: raise() 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: abort() 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x563c19dac237] 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 7: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 8: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.376 INFO:tasks.ceph.osd.5.smithi153.stderr: 9: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.377 INFO:tasks.ceph.osd.5.smithi153.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T03:29:00.377 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr: -5> 2024-05-27T03:29:00.369+0000 7f76d59a5640 -1 bdev(0x563c1cb3d180 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr: -4> 2024-05-27T03:29:00.370+0000 7f76d59a5640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f76d59a5640 time 2024-05-27T03:29:00.370829+0000 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x563c19dac176] 2024-05-27T03:29:00.389 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 0> 2024-05-27T03:29:00.372+0000 7f76d59a5640 -1 *** Caught signal (Aborted) ** 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: in thread 7f76d59a5640 thread_name:bstore_aio 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76e303e6f0] 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f76e308b94c] 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: raise() 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: abort() 2024-05-27T03:29:00.390 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x563c19dac237] 2024-05-27T03:29:00.391 INFO:tasks.ceph.osd.5.smithi153.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.391 INFO:tasks.ceph.osd.5.smithi153.stderr: 7: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.391 INFO:tasks.ceph.osd.5.smithi153.stderr: 8: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.391 INFO:tasks.ceph.osd.5.smithi153.stderr: 9: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.391 INFO:tasks.ceph.osd.5.smithi153.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T03:29:00.391 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: -210> 2024-05-27T03:29:00.369+0000 7f76d59a5640 -1 bdev(0x563c1cb3d180 /var/lib/ceph/osd/ceph-5/block) _aio_thread got r=-28 ((28) No space left on device) 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: -209> 2024-05-27T03:29:00.370+0000 7f76d59a5640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f76d59a5640 time 2024-05-27T03:29:00.370829+0000 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2857-g0ef46a24/rpm/el9/BUILD/ceph-19.0.0-2857-g0ef46a24/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x563c19dac176] 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.399 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: -205> 2024-05-27T03:29:00.372+0000 7f76d59a5640 -1 *** Caught signal (Aborted) ** 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: in thread 7f76d59a5640 thread_name:bstore_aio 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: ceph version 19.0.0-2857-g0ef46a24 (0ef46a246e1118e5adafa1857b4189e7d780c816) squid (rc) 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76e303e6f0] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f76e308b94c] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 3: raise() 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 4: abort() 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x563c19dac237] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x563c1a7748e0] 2024-05-27T03:29:00.400 INFO:tasks.ceph.osd.5.smithi153.stderr: 7: ceph-osd(+0xdb8a51) [0x563c1a774a51] 2024-05-27T03:29:00.401 INFO:tasks.ceph.osd.5.smithi153.stderr: 8: /lib64/libc.so.6(+0x89c02) [0x7f76e3089c02] 2024-05-27T03:29:00.401 INFO:tasks.ceph.osd.5.smithi153.stderr: 9: /lib64/libc.so.6(+0x10ec40) [0x7f76e310ec40] 2024-05-27T03:29:00.401 INFO:tasks.ceph.osd.5.smithi153.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T03:29:00.401 INFO:tasks.ceph.osd.5.smithi153.stderr: 2024-05-27T03:29:00.544 INFO:tasks.ceph.osd.5.smithi153.stderr:daemon-helper: command failed with exit status 5 2024-05-27T03:29:05.087 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-27T03:29:11.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~7s 2024-05-27T03:29:18.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~13s 2024-05-27T03:29:24.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-05-27T03:29:31.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~26s 2024-05-27T03:29:38.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~33s 2024-05-27T03:29:44.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-05-27T03:29:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~46s 2024-05-27T03:29:57.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~53s 2024-05-27T03:30:04.584 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~59s 2024-05-27T03:30:11.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~66s 2024-05-27T03:30:17.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~73s 2024-05-27T03:30:24.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~79s 2024-05-27T03:30:31.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~86s 2024-05-27T03:30:37.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~93s 2024-05-27T03:30:44.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~99s 2024-05-27T03:30:50.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~106s 2024-05-27T03:30:57.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~112s 2024-05-27T03:31:04.085 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~119s 2024-05-27T03:31:10.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~126s 2024-05-27T03:31:17.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~132s 2024-05-27T03:31:23.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~139s 2024-05-27T03:31:30.537 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~145s 2024-05-27T03:31:32.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:31:32.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:31:32.641 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T03:31:32.640+0000 7f6764b08640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50761) UID: 0 2024-05-27T03:31:32.642 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T03:31:32.640+0000 7f62b9de8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50761) UID: 0 2024-05-27T03:31:32.642 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T03:31:32.640+0000 7f2307fdb640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50761) UID: 0 2024-05-27T03:31:32.710 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T03:31:32.710+0000 7fb6912fe640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595412) UID: 0 2024-05-27T03:31:32.711 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T03:31:32.710+0000 7f3531e31640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595412) UID: 0 2024-05-27T03:31:32.711 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T03:31:32.711+0000 7f2e7fb39640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 595412) UID: 0 2024-05-27T03:31:32.711 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:31:32.711+0000 7fcaa45a7640 -1 Fail to open '/proc/595412/cmdline' error = (2) No such file or directory 2024-05-27T03:31:32.712 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:31:32.711+0000 7fcaa45a7640 -1 received signal: Hangup from (PID: 595412) UID: 0 2024-05-27T03:31:37.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~152s 2024-05-27T03:31:43.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~159s 2024-05-27T03:31:50.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~165s 2024-05-27T03:31:56.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~172s 2024-05-27T03:32:03.606 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~179s 2024-05-27T03:32:10.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2024-05-27T03:32:16.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~192s 2024-05-27T03:32:23.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~198s 2024-05-27T03:32:30.058 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2024-05-27T03:32:36.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~212s 2024-05-27T03:32:43.284 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~218s 2024-05-27T03:32:49.897 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~225s 2024-05-27T03:32:56.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~231s 2024-05-27T03:33:03.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~238s 2024-05-27T03:33:09.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~245s 2024-05-27T03:33:16.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~251s 2024-05-27T03:33:22.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~258s 2024-05-27T03:33:29.574 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~264s 2024-05-27T03:33:36.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~271s 2024-05-27T03:33:42.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~278s 2024-05-27T03:33:49.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~284s 2024-05-27T03:33:56.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~291s 2024-05-27T03:34:02.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~298s 2024-05-27T03:34:09.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~304s 2024-05-27T03:34:09.252 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-05-27T03:34:09.253 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T03:34:09.253 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T03:34:09.292 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T03:34:09.293 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T03:34:09.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T03:34:11.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.115+0000 7fd170a8c640 1 Processor -- start 2024-05-27T03:34:11.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.116+0000 7fd170a8c640 1 -- start start 2024-05-27T03:34:11.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd170a8c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd16c14b4b0 0x7fd16c071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd170a8c640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd16c06c780 0x7fd16c0719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd170a8c640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd16c0b3cf0 con 0x7fd16c14c190 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd170a8c640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fd16c0b3fd0 con 0x7fd16c06c780 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd170a8c640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd16c0b42b0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd16a59c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd16c14b4b0 0x7fd16c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd16a59c640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd16c14b4b0 0x7fd16c071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:50530/0 (socket says 172.21.15.111:50530) 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.127+0000 7fd16a59c640 1 -- 172.21.15.111:0/2558075810 learned_addr learned my addr 172.21.15.111:0/2558075810 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:11.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.128+0000 7fd16a59c640 1 -- 172.21.15.111:0/2558075810 >> v2:172.21.15.111:3300/0 conn(0x7fd16c06c780 msgr2=0x7fd16c0719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:11.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.128+0000 7fd16a59c640 1 --2- 172.21.15.111:0/2558075810 >> v2:172.21.15.111:3300/0 conn(0x7fd16c06c780 0x7fd16c0719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.128+0000 7fd16a59c640 1 -- 172.21.15.111:0/2558075810 >> v1:172.21.15.111:6789/0 conn(0x7fd16c14c190 legacy=0x7fd16c166200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.128+0000 7fd16a59c640 1 -- 172.21.15.111:0/2558075810 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c0b4590 con 0x7fd16c14b4b0 2024-05-27T03:34:11.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.129+0000 7fd16a59c640 1 --2- 172.21.15.111:0/2558075810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14b4b0 0x7fd16c071470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd16c06fb30 tx=0x7fd15c02a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e8331d5d4a49e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:11.130 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.130+0000 7fd169d9b640 1 -- 172.21.15.111:0/2558075810 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c037070 con 0x7fd16c14b4b0 2024-05-27T03:34:11.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.139+0000 7fd169d9b640 1 -- 172.21.15.111:0/2558075810 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c032440 con 0x7fd16c14b4b0 2024-05-27T03:34:11.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.139+0000 7fd169d9b640 1 -- 172.21.15.111:0/2558075810 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c032810 con 0x7fd16c14b4b0 2024-05-27T03:34:11.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.140+0000 7fd170a8c640 1 -- 172.21.15.111:0/2558075810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14b4b0 msgr2=0x7fd16c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.140+0000 7fd170a8c640 1 --2- 172.21.15.111:0/2558075810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14b4b0 0x7fd16c071470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd16c06fb30 tx=0x7fd15c02a1f0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.140+0000 7fd170a8c640 1 -- 172.21.15.111:0/2558075810 shutdown_connections 2024-05-27T03:34:11.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.140+0000 7fd170a8c640 1 --2- 172.21.15.111:0/2558075810 >> v2:172.21.15.111:3300/0 conn(0x7fd16c06c780 0x7fd16c0719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.140+0000 7fd170a8c640 1 --2- 172.21.15.111:0/2558075810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14b4b0 0x7fd16c071470 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.140+0000 7fd170a8c640 1 -- 172.21.15.111:0/2558075810 >> 172.21.15.111:0/2558075810 conn(0x7fd16c0cfe50 msgr2=0x7fd16c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:11.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.141+0000 7fd170a8c640 1 -- 172.21.15.111:0/2558075810 shutdown_connections 2024-05-27T03:34:11.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.141+0000 7fd170a8c640 1 -- 172.21.15.111:0/2558075810 wait complete. 2024-05-27T03:34:11.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.143+0000 7fd170a8c640 1 Processor -- start 2024-05-27T03:34:11.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.143+0000 7fd170a8c640 1 -- start start 2024-05-27T03:34:11.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd170a8c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 0x7fd16c147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd170a8c640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd16c14b4b0 0x7fd16c15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd170a8c640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14c190 0x7fd16c1ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd16a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 0x7fd16c147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd16a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 0x7fd16c147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38130/0 (socket says 172.21.15.111:38130) 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd16a59c640 1 -- 172.21.15.111:0/403713308 learned_addr learned my addr 172.21.15.111:0/403713308 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd170a8c640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0b3a90 con 0x7fd16c06c780 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd170a8c640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0b3910 con 0x7fd16c14c190 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.144+0000 7fd170a8c640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c14bbd0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd16c14b4b0 0x7fd16c15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd16ad9d640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14c190 0x7fd16c1ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 -- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14c190 msgr2=0x7fd16c1ed7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14c190 0x7fd16c1ed7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 -- 172.21.15.111:0/403713308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 msgr2=0x7fd16c147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 0x7fd16c147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c15eae0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd16a59c640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 0x7fd16c147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T03:34:11.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.145+0000 7fd163fff640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd16c14b4b0 0x7fd16c15e2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c390 tx=0x7fd15400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:11.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.160+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154017070 con 0x7fd16c14b4b0 2024-05-27T03:34:11.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.160+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1540052d0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.160+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154003830 con 0x7fd16c14b4b0 2024-05-27T03:34:11.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.160+0000 7fd170a8c640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c15edc0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.160+0000 7fd170a8c640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c160880 con 0x7fd16c14b4b0 2024-05-27T03:34:11.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.164+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd150005290 con 0x7fd16c14b4b0 2024-05-27T03:34:11.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.167+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd15401a070 con 0x7fd16c14b4b0 2024-05-27T03:34:11.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.168+0000 7fd162ffd640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd14c06be80 0x7fd14c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.168+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 6623+0+0 (secure 0 0 0) 0x7fd154055250 con 0x7fd16c14b4b0 2024-05-27T03:34:11.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.168+0000 7fd16a59c640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd14c06be80 0x7fd14c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.169+0000 7fd16a59c640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd14c06be80 0x7fd14c06e340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0090b0 tx=0x7fd15c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:11.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.184+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd154058be0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.499 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.499+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd150005850 con 0x7fd16c14b4b0 2024-05-27T03:34:11.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.501+0000 7fd162ffd640 1 -- 172.21.15.111:0/403713308 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17751 (secure 0 0 0) 0x7fd15405cee0 con 0x7fd16c14b4b0 2024-05-27T03:34:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-05-27T03:34:11.505 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":51,"fsid":"ef97d1dc-a8f7-4487-922a-ccc176f6927a","created":"2024-05-27T00:38:56.598102+0000","modified":"2024-05-27T03:29:16.152491+0000","last_up_change":"2024-05-27T03:29:00.637039+0000","last_in_change":"2024-05-27T00:38:58.939697+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T00:39:08.955617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-27T00:40:28.415554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-27T00:40:29.147789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bddbbc32-41a7-4d6c-b42e-a7c7b6c1638d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6818","nonce":4118349595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6822","nonce":4118349595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6830","nonce":4118349595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4118349595},{"type":"v1","addr":"172.21.15.111:6826","nonce":4118349595}]},"public_addr":"172.21.15.111:6818/4118349595","cluster_addr":"172.21.15.111:6822/4118349595","heartbeat_back_addr":"172.21.15.111:6830/4118349595","heartbeat_front_addr":"172.21.15.111:6826/4118349595","state":["exists","up"]},{"osd":1,"uuid":"66cc4ce4-d74b-4952-8f95-f6b7d2c1c562","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6801","nonce":2862511107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6803","nonce":2862511107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6807","nonce":2862511107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2862511107},{"type":"v1","addr":"172.21.15.111:6805","nonce":2862511107}]},"public_addr":"172.21.15.111:6801/2862511107","cluster_addr":"172.21.15.111:6803/2862511107","heartbeat_back_addr":"172.21.15.111:6807/2862511107","heartbeat_front_addr":"172.21.15.111:6805/2862511107","state":["exists","up"]},{"osd":2,"uuid":"83c870e7-b431-4b2e-9cc1-dd7d235821c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6809","nonce":1204898981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6811","nonce":1204898981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6815","nonce":1204898981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":1204898981},{"type":"v1","addr":"172.21.15.111:6813","nonce":1204898981}]},"public_addr":"172.21.15.111:6809/1204898981","cluster_addr":"172.21.15.111:6811/1204898981","heartbeat_back_addr":"172.21.15.111:6815/1204898981","heartbeat_front_addr":"172.21.15.111:6813/1204898981","state":["exists","up"]},{"osd":3,"uuid":"34090321-f9cd-47ea-bfce-db5326e5d4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6819","nonce":1957218503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6821","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6823","nonce":1957218503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6829","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6831","nonce":1957218503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6825","nonce":1957218503},{"type":"v1","addr":"172.21.15.111:6827","nonce":1957218503}]},"public_addr":"172.21.15.111:6819/1957218503","cluster_addr":"172.21.15.111:6823/1957218503","heartbeat_back_addr":"172.21.15.111:6831/1957218503","heartbeat_front_addr":"172.21.15.111:6827/1957218503","state":["exists","up"]},{"osd":4,"uuid":"7fbcce4c-7fd0-4eac-a8ad-5c386795cc5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6809","nonce":3644921955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6811","nonce":3644921955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6815","nonce":3644921955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3644921955},{"type":"v1","addr":"172.21.15.153:6813","nonce":3644921955}]},"public_addr":"172.21.15.153:6809/3644921955","cluster_addr":"172.21.15.153:6811/3644921955","heartbeat_back_addr":"172.21.15.153:6815/3644921955","heartbeat_front_addr":"172.21.15.153:6813/3644921955","state":["exists","up"]},{"osd":5,"uuid":"1be914ee-b413-449c-8453-a55c28af7d70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":42,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6801","nonce":668441948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6803","nonce":668441948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6807","nonce":668441948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":668441948},{"type":"v1","addr":"172.21.15.153:6805","nonce":668441948}]},"public_addr":"172.21.15.153:6801/668441948","cluster_addr":"172.21.15.153:6803/668441948","heartbeat_back_addr":"172.21.15.153:6807/668441948","heartbeat_front_addr":"172.21.15.153:6805/668441948","state":["exists"]},{"osd":6,"uuid":"c88ab04d-9802-4926-b3e4-38a8d055e63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6825","nonce":3277363095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6827","nonce":3277363095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6831","nonce":3277363095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3277363095},{"type":"v1","addr":"172.21.15.153:6829","nonce":3277363095}]},"public_addr":"172.21.15.153:6825/3277363095","cluster_addr":"172.21.15.153:6827/3277363095","heartbeat_back_addr":"172.21.15.153:6831/3277363095","heartbeat_front_addr":"172.21.15.153:6829/3277363095","state":["exists","up"]},{"osd":7,"uuid":"1356774a-a6a9-4ce5-be06-a153448df9db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6817","nonce":2433121395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6819","nonce":2433121395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6823","nonce":2433121395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2433121395},{"type":"v1","addr":"172.21.15.153:6821","nonce":2433121395}]},"public_addr":"172.21.15.153:6817/2433121395","cluster_addr":"172.21.15.153:6819/2433121395","heartbeat_back_addr":"172.21.15.153:6823/2433121395","heartbeat_front_addr":"172.21.15.153:6821/2433121395","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.108574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.049052+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.109601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.128607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.006131+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.080493+0000","dead_epoch":42},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.124159+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T00:39:04.097841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.5","mappings":[{"from":1,"to":5}]},{"pgid":"5.c","mappings":[{"from":4,"to":2}]},{"pgid":"5.f","mappings":[{"from":4,"to":6}]},{"pgid":"5.10","mappings":[{"from":4,"to":5}]},{"pgid":"5.1c","mappings":[{"from":4,"to":5}]},{"pgid":"6.2","mappings":[{"from":3,"to":5}]},{"pgid":"6.9","mappings":[{"from":7,"to":5}]},{"pgid":"6.20","mappings":[{"from":2,"to":5}]},{"pgid":"6.24","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"5.5","osds":[0,1]},{"pgid":"5.f","osds":[6,4]},{"pgid":"5.10","osds":[7,4]},{"pgid":"5.1c","osds":[3,4]},{"pgid":"6.2","osds":[4,3]},{"pgid":"6.9","osds":[0,7]},{"pgid":"6.20","osds":[1,2]},{"pgid":"6.24","osds":[0,3]},{"pgid":"6.38","osds":[3,0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T03:34:11.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.507+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd14c06be80 msgr2=0x7fd14c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.507+0000 7fd160ff9640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd14c06be80 0x7fd14c06e340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0090b0 tx=0x7fd15c02e000 comp rx=0 tx=0).stop 2024-05-27T03:34:11.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.507+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd16c14b4b0 msgr2=0x7fd16c15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.507 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.507+0000 7fd160ff9640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd16c14b4b0 0x7fd16c15e2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c390 tx=0x7fd15400c850 comp rx=0 tx=0).stop 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 shutdown_connections 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd14c06be80 0x7fd14c06e340 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd16c14c190 0x7fd16c1ed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd16c14b4b0 0x7fd16c15e2b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 --2- 172.21.15.111:0/403713308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd16c06c780 0x7fd16c147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 >> 172.21.15.111:0/403713308 conn(0x7fd16c0cfe50 msgr2=0x7fd16c14a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 shutdown_connections 2024-05-27T03:34:11.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.508+0000 7fd160ff9640 1 -- 172.21.15.111:0/403713308 wait complete. 2024-05-27T03:34:11.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T00:39:08.955617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T00:40:28.415554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T00:40:29.147789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T03:34:11.528 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T03:34:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.702+0000 7fb91dac3640 1 Processor -- start 2024-05-27T03:34:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.702+0000 7fb91dac3640 1 -- start start 2024-05-27T03:34:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.703+0000 7fb91dac3640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb91814b4b0 0x7fb918071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.703+0000 7fb91dac3640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb918167020 con 0x7fb91814c190 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.703+0000 7fb91dac3640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9180b3cf0 con 0x7fb91806c780 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.703+0000 7fb91dac3640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7fb9180b3fd0 con 0x7fb91814b4b0 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb917fff640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb91806c780 0x7fb918166910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:50734/0 (socket says 172.21.15.111:50734) 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb916ffd640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb91814c190 0x7fb918166200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38548/0 (socket says 172.21.15.111:38548) 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb917fff640 1 -- 172.21.15.111:0/402471378 learned_addr learned my addr 172.21.15.111:0/402471378 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9177fe640 1 --2- 172.21.15.111:0/402471378 >> v2:172.21.15.111:3300/0 conn(0x7fb91814b4b0 0x7fb918071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.1 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079995075 0 0) 0x7fb9180b3cf0 con 0x7fb91806c780 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb904003520 con 0x7fb91806c780 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2654429411 0 0) 0x7fb918167020 con 0x7fb91814c190 2024-05-27T03:34:11.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9180b3cf0 con 0x7fb91814c190 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.1 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3816094565 0 0) 0x7fb904003520 con 0x7fb91806c780 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 >> v2:172.21.15.111:3300/0 conn(0x7fb91814b4b0 msgr2=0x7fb918071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 --2- 172.21.15.111:0/402471378 >> v2:172.21.15.111:3300/0 conn(0x7fb91814b4b0 0x7fb918071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 >> v1:172.21.15.111:6789/0 conn(0x7fb91814c190 legacy=0x7fb918166200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9180b42b0 con 0x7fb91806c780 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.1 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7fb9100019f0 con 0x7fb91806c780 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.704+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 >> v1:172.21.15.153:6789/0 conn(0x7fb91806c780 legacy=0x7fb918166910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9167fc640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb904004bf0 0x7fb904004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9167fc640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9040037a0 0x7fb904003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9167fc640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb90400ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.706 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9180b3fd0 con 0x7fb9040037a0 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb90400c190 con 0x7fb904004bf0 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb90400c390 con 0x7fb904004160 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb917fff640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb90400ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb9177fe640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb904004bf0 0x7fb904004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb917fff640 1 -- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb904004bf0 msgr2=0x7fb904004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb917fff640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb904004bf0 0x7fb904004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb917fff640 1 -- 172.21.15.111:0/402471378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9040037a0 msgr2=0x7fb904003c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb917fff640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9040037a0 0x7fb904003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.705+0000 7fb917fff640 1 -- 172.21.15.111:0/402471378 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb904003520 con 0x7fb904004160 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.706+0000 7fb917fff640 1 -- 172.21.15.111:0/402471378 reap_dead start 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.706+0000 7fb917fff640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb90400ba40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb91000a750 tx=0x7fb904005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c20c99440a157131 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.707+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91000a370 con 0x7fb904004160 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.707+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91002f070 con 0x7fb904004160 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.707+0000 7fb9167fc640 1 -- 172.21.15.111:0/402471378 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91002bb90 con 0x7fb904004160 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 -- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 msgr2=0x7fb90400ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.708 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb90400ba40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb91000a750 tx=0x7fb904005830 comp rx=0 tx=0).stop 2024-05-27T03:34:11.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 -- 172.21.15.111:0/402471378 shutdown_connections 2024-05-27T03:34:11.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 --2- 172.21.15.111:0/402471378 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb90400ba40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 -- 172.21.15.111:0/402471378 >> 172.21.15.111:0/402471378 conn(0x7fb9180cfe50 msgr2=0x7fb9180d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:11.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 -- 172.21.15.111:0/402471378 shutdown_connections 2024-05-27T03:34:11.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.708+0000 7fb91dac3640 1 -- 172.21.15.111:0/402471378 wait complete. 2024-05-27T03:34:11.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.710+0000 7fb91dac3640 1 Processor -- start 2024-05-27T03:34:11.713 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.712+0000 7fb91dac3640 1 -- start start 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb91dac3640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb91814a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb91dac3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb918157260 0x7fb918148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb91dac3640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918146ee0 0x7fb918135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb91dac3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb91814ba10 con 0x7fb918157260 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb91dac3640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fb91814b890 con 0x7fb918146ee0 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb91dac3640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fb9180b3910 con 0x7fb904004160 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb9177fe640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb91814a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb916ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb918157260 0x7fb918148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb916ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb918157260 0x7fb918148c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38150/0 (socket says 172.21.15.111:38150) 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb916ffd640 1 -- 172.21.15.111:0/3427871799 learned_addr learned my addr 172.21.15.111:0/3427871799 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:11.714 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.713+0000 7fb917fff640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918146ee0 0x7fb918135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.714+0000 7fb9177fe640 1 -- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918146ee0 msgr2=0x7fb918135950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.714+0000 7fb9177fe640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918146ee0 0x7fb918135950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.714+0000 7fb9177fe640 1 -- 172.21.15.111:0/3427871799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb918157260 msgr2=0x7fb918148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.714+0000 7fb9177fe640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb918157260 0x7fb918148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.714+0000 7fb9177fe640 1 -- 172.21.15.111:0/3427871799 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb918146720 con 0x7fb904004160 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.714+0000 7fb9177fe640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb91814a810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9000115c0 tx=0x7fb900011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:11.715 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.715+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900019070 con 0x7fb904004160 2024-05-27T03:34:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.715+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb918157b00 con 0x7fb904004160 2024-05-27T03:34:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.715+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181580c0 con 0x7fb904004160 2024-05-27T03:34:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.715+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb900003610 con 0x7fb904004160 2024-05-27T03:34:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.716+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000039c0 con 0x7fb904004160 2024-05-27T03:34:11.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.716+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc005290 con 0x7fb904004160 2024-05-27T03:34:11.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.718+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb90001c070 con 0x7fb904004160 2024-05-27T03:34:11.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.718+0000 7fb90ffff640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8f806bf30 0x7fb8f806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:11.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.718+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 6623+0+0 (secure 0 0 0) 0x7fb90008b0c0 con 0x7fb904004160 2024-05-27T03:34:11.719 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.719+0000 7fb916ffd640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8f806bf30 0x7fb8f806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:11.720 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.720+0000 7fb916ffd640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8f806bf30 0x7fb8f806e3f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb908007fd0 tx=0x7fb908007d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:11.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:11.727+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb900059070 con 0x7fb904004160 2024-05-27T03:34:12.056 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.056+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8fc005d90 con 0x7fb904004160 2024-05-27T03:34:12.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.057+0000 7fb90ffff640 1 -- 172.21.15.111:0/3427871799 <== mon.2 v2:172.21.15.153:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7fb90005d370 con 0x7fb904004160 2024-05-27T03:34:12.059 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-05-27T03:34:12.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.061+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8f806bf30 msgr2=0x7fb8f806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.061+0000 7fb91dac3640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8f806bf30 0x7fb8f806e3f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb908007fd0 tx=0x7fb908007d60 comp rx=0 tx=0).stop 2024-05-27T03:34:12.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.061+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 msgr2=0x7fb91814a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.061+0000 7fb91dac3640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb91814a810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9000115c0 tx=0x7fb900011a80 comp rx=0 tx=0).stop 2024-05-27T03:34:12.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.063+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 shutdown_connections 2024-05-27T03:34:12.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.063+0000 7fb91dac3640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fb8f806bf30 0x7fb8f806e3f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.063+0000 7fb91dac3640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918146ee0 0x7fb918135950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.063+0000 7fb91dac3640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb918157260 0x7fb918148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.063+0000 7fb91dac3640 1 --2- 172.21.15.111:0/3427871799 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb904004160 0x7fb91814a810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.063+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 >> 172.21.15.111:0/3427871799 conn(0x7fb9180cfe50 msgr2=0x7fb918147a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:12.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.066+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 shutdown_connections 2024-05-27T03:34:12.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.066+0000 7fb91dac3640 1 -- 172.21.15.111:0/3427871799 wait complete. 2024-05-27T03:34:12.092 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T03:34:12.267 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.267+0000 7f98f1027640 1 Processor -- start 2024-05-27T03:34:12.268 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.267+0000 7f98f1027640 1 -- start start 2024-05-27T03:34:12.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.268+0000 7f98f1027640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec144830 0x7f98ec0d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.268+0000 7f98f1027640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec144c10 con 0x7f98ec070a10 2024-05-27T03:34:12.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98f1027640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec06f510 con 0x7f98ec0705b0 2024-05-27T03:34:12.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98f1027640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec0d0ae0 con 0x7f98ec144830 2024-05-27T03:34:12.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98eb59e640 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f98ec070a10 0x7f98ec15b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38552/0 (socket says 172.21.15.111:38552) 2024-05-27T03:34:12.269 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98eb59e640 1 -- 172.21.15.111:0/3804062732 learned_addr learned my addr 172.21.15.111:0/3804062732 (peer_addr_for_me v1:172.21.15.111:0/0) 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98ead9d640 1 --2- 172.21.15.111:0/3804062732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec144830 0x7f98ec0d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98e9d9b640 1 -- 172.21.15.111:0/3804062732 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268374117 0 0) 0x7f98ec144c10 con 0x7f98ec070a10 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98e9d9b640 1 -- 172.21.15.111:0/3804062732 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98d4003520 con 0x7f98ec070a10 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98ead9d640 1 -- 172.21.15.111:0/3804062732 >> v1:172.21.15.153:6789/0 conn(0x7f98ec0705b0 legacy=0x7f98ec0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98ead9d640 1 -- 172.21.15.111:0/3804062732 >> v1:172.21.15.111:6789/0 conn(0x7f98ec070a10 legacy=0x7f98ec15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.269+0000 7f98ead9d640 1 -- 172.21.15.111:0/3804062732 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec06f7f0 con 0x7f98ec144830 2024-05-27T03:34:12.270 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.270+0000 7f98ead9d640 1 --2- 172.21.15.111:0/3804062732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec144830 0x7f98ec0d05a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009e50 tx=0x7f98e402c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293e91baf0a97ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:12.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.271+0000 7f98e9d9b640 1 -- 172.21.15.111:0/3804062732 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4036020 con 0x7f98ec144830 2024-05-27T03:34:12.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.271+0000 7f98e9d9b640 1 -- 172.21.15.111:0/3804062732 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e4032820 con 0x7f98ec144830 2024-05-27T03:34:12.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.271+0000 7f98e9d9b640 1 -- 172.21.15.111:0/3804062732 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4032c30 con 0x7f98ec144830 2024-05-27T03:34:12.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.272+0000 7f98f1027640 1 -- 172.21.15.111:0/3804062732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec144830 msgr2=0x7f98ec0d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.272+0000 7f98f1027640 1 --2- 172.21.15.111:0/3804062732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec144830 0x7f98ec0d05a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009e50 tx=0x7f98e402c960 comp rx=0 tx=0).stop 2024-05-27T03:34:12.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.272+0000 7f98f1027640 1 -- 172.21.15.111:0/3804062732 shutdown_connections 2024-05-27T03:34:12.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.272+0000 7f98f1027640 1 --2- 172.21.15.111:0/3804062732 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec144830 0x7f98ec0d05a0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009e50 tx=0x7f98e402c960 comp rx=0 tx=0).stop 2024-05-27T03:34:12.272 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.272+0000 7f98f1027640 1 -- 172.21.15.111:0/3804062732 >> 172.21.15.111:0/3804062732 conn(0x7f98ec072250 msgr2=0x7f98ec072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:12.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.273+0000 7f98f1027640 1 -- 172.21.15.111:0/3804062732 shutdown_connections 2024-05-27T03:34:12.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.273+0000 7f98f1027640 1 -- 172.21.15.111:0/3804062732 wait complete. 2024-05-27T03:34:12.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.275+0000 7f98f1027640 1 Processor -- start 2024-05-27T03:34:12.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.275+0000 7f98f1027640 1 -- start start 2024-05-27T03:34:12.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.276+0000 7f98f1027640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec0705b0 0x7f98ec0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.276+0000 7f98f1027640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98ec1f3320 0x7f98ec15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.277 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.276+0000 7f98f1027640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 0x7f98ec13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98ead9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec0705b0 0x7f98ec0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98ead9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec0705b0 0x7f98ec0ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:42950/0 (socket says 172.21.15.111:42950) 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98f1027640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec06fdb0 con 0x7f98ec15a980 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98f1027640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec070120 con 0x7f98ec1f3320 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98f1027640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec06cf60 con 0x7f98ec0705b0 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98eb59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 0x7f98ec13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.279 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98eb59e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 0x7f98ec13fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38154/0 (socket says 172.21.15.111:38154) 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98eb59e640 1 -- 172.21.15.111:0/4142277997 learned_addr learned my addr 172.21.15.111:0/4142277997 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.277+0000 7f98ea59c640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98ec1f3320 0x7f98ec15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.278+0000 7f98eb59e640 1 -- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec0705b0 msgr2=0x7f98ec0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.278+0000 7f98eb59e640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec0705b0 0x7f98ec0ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.278+0000 7f98eb59e640 1 -- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98ec1f3320 msgr2=0x7f98ec15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.278+0000 7f98eb59e640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98ec1f3320 0x7f98ec15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.278+0000 7f98eb59e640 1 -- 172.21.15.111:0/4142277997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0ace60 con 0x7f98ec15a980 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.278+0000 7f98ea59c640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98ec1f3320 0x7f98ec15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.279+0000 7f98eb59e640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 0x7f98ec13fdc0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c3c0 tx=0x7f98e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:12.280 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.279+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0017070 con 0x7f98ec15a980 2024-05-27T03:34:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.279+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec0ad140 con 0x7f98ec15a980 2024-05-27T03:34:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.280+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec0acb70 con 0x7f98ec15a980 2024-05-27T03:34:12.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.281+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec0ad660 con 0x7f98ec15a980 2024-05-27T03:34:12.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.318+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e0005ce0 con 0x7f98ec15a980 2024-05-27T03:34:12.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.318+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0021060 con 0x7f98ec15a980 2024-05-27T03:34:12.345 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.345+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98e001a070 con 0x7f98ec15a980 2024-05-27T03:34:12.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.345+0000 7f98d37fe640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98d806be30 0x7f98d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.345+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6623+0+0 (secure 0 0 0) 0x7f98e008ac10 con 0x7f98ec15a980 2024-05-27T03:34:12.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.345+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f98ec0ad660 con 0x7f98ec15a980 2024-05-27T03:34:12.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.346+0000 7f98ead9d640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98d806be30 0x7f98d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.346 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.346+0000 7f98ead9d640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98d806be30 0x7f98d806e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f98e4037000 tx=0x7f98e4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:12.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.699+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98ec068c20 con 0x7f98ec15a980 2024-05-27T03:34:12.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.700+0000 7f98d37fe640 1 -- 172.21.15.111:0/4142277997 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) ==== 104+0+11 (secure 0 0 0) 0x7f98ec068c20 con 0x7f98ec15a980 2024-05-27T03:34:12.701 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-05-27T03:34:12.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.703+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98d806be30 msgr2=0x7f98d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98d806be30 0x7f98d806e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f98e4037000 tx=0x7f98e4032000 comp rx=0 tx=0).stop 2024-05-27T03:34:12.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 msgr2=0x7f98ec13fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 0x7f98ec13fdc0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c3c0 tx=0x7f98e000c880 comp rx=0 tx=0).stop 2024-05-27T03:34:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 shutdown_connections 2024-05-27T03:34:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7f98d806be30 0x7f98d806e2f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98ec15a980 0x7f98ec13fdc0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98ec1f3320 0x7f98ec15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 --2- 172.21.15.111:0/4142277997 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7f98ec0705b0 0x7f98ec0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.704+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 >> 172.21.15.111:0/4142277997 conn(0x7f98ec072250 msgr2=0x7f98ec0ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:12.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.710+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 shutdown_connections 2024-05-27T03:34:12.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.710+0000 7f98f1027640 1 -- 172.21.15.111:0/4142277997 wait complete. 2024-05-27T03:34:12.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T03:34:12.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.902+0000 7efd7c9e8640 1 Processor -- start 2024-05-27T03:34:12.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.902+0000 7efd7c9e8640 1 -- start start 2024-05-27T03:34:12.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7c9e8640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efd780d0a00 0x7efd781468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7c9e8640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd78070680 0x7efd78146e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7c9e8640 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780a9b10 con 0x7efd780702a0 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7c9e8640 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 -- 0x7efd780b5b60 con 0x7efd78070680 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7c9e8640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7efd780b5e40 con 0x7efd780d0a00 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7759e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd78070680 0x7efd78146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7759e640 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd78070680 0x7efd78146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38164/0 (socket says 172.21.15.111:38164) 2024-05-27T03:34:12.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.904+0000 7efd7759e640 1 -- 172.21.15.111:0/1637312430 learned_addr learned my addr 172.21.15.111:0/1637312430 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2561750583 0 0) 0x7efd780a9b10 con 0x7efd780702a0 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003520 con 0x7efd780702a0 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 369396156 0 0) 0x7efd64003520 con 0x7efd780702a0 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 >> v2:172.21.15.153:3300/0 conn(0x7efd780d0a00 msgr2=0x7efd781468f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 --2- 172.21.15.111:0/1637312430 >> v2:172.21.15.153:3300/0 conn(0x7efd780d0a00 0x7efd781468f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 >> v2:172.21.15.111:3300/0 conn(0x7efd78070680 msgr2=0x7efd78146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 --2- 172.21.15.111:0/1637312430 >> v2:172.21.15.111:3300/0 conn(0x7efd78070680 0x7efd78146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780b6120 con 0x7efd780702a0 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4253864261 0 0) 0x7efd68001670 con 0x7efd780702a0 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 >> v1:172.21.15.111:6789/0 conn(0x7efd780702a0 legacy=0x7efd780a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.905+0000 7efd7659c640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.906+0000 7efd7659c640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd64004800 0x7efd64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.906+0000 7efd7659c640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd640051c0 0x7efd640141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.906+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd780b5b60 con 0x7efd64004800 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.906+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd780b5e40 con 0x7efd640051c0 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.906+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7efd640148f0 con 0x7efd640039b0 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.906+0000 7efd76d9d640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd7759e640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd640051c0 0x7efd640141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd6ffff640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd64004800 0x7efd64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd76d9d640 1 -- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd640051c0 msgr2=0x7efd640141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd76d9d640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd640051c0 0x7efd640141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd76d9d640 1 -- 172.21.15.111:0/1637312430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd64004800 msgr2=0x7efd64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd76d9d640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd64004800 0x7efd64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.907+0000 7efd76d9d640 1 -- 172.21.15.111:0/1637312430 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780b6120 con 0x7efd640039b0 2024-05-27T03:34:12.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.908+0000 7efd6ffff640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd64004800 0x7efd64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T03:34:12.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.908+0000 7efd7759e640 1 -- 172.21.15.111:0/1637312430 reap_dead start 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.908+0000 7efd76d9d640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd64003dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c1b0 tx=0x7efd64005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b52b8dde28326332 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.908+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 <== mon.2 v2:172.21.15.153:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68036070 con 0x7efd640039b0 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.909+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 <== mon.2 v2:172.21.15.153:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68031620 con 0x7efd640039b0 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.909+0000 7efd7659c640 1 -- 172.21.15.111:0/1637312430 <== mon.2 v2:172.21.15.153:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68007ca0 con 0x7efd640039b0 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.909+0000 7efd7c9e8640 1 -- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 msgr2=0x7efd64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.909+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd64003dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c1b0 tx=0x7efd64005830 comp rx=0 tx=0).stop 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.910+0000 7efd7c9e8640 1 -- 172.21.15.111:0/1637312430 shutdown_connections 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.910+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/1637312430 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd64003dd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.910+0000 7efd7c9e8640 1 -- 172.21.15.111:0/1637312430 >> 172.21.15.111:0/1637312430 conn(0x7efd780723e0 msgr2=0x7efd780727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.910+0000 7efd7c9e8640 1 -- 172.21.15.111:0/1637312430 shutdown_connections 2024-05-27T03:34:12.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.910+0000 7efd7c9e8640 1 -- 172.21.15.111:0/1637312430 wait complete. 2024-05-27T03:34:12.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.912+0000 7efd7c9e8640 1 Processor -- start 2024-05-27T03:34:12.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.912+0000 7efd7c9e8640 1 -- start start 2024-05-27T03:34:12.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd7c9e8640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd7814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd7c9e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd7815db70 0x7efd781489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd76d9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd7814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd76d9d640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd7814be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3301/0 says I am v2:172.21.15.111:42978/0 (socket says 172.21.15.111:42978) 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd76d9d640 1 -- 172.21.15.111:0/4063950118 learned_addr learned my addr 172.21.15.111:0/4063950118 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd78148f00 0x7efd781435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd780a9400 con 0x7efd7815db70 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.914+0000 7efd7759e640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd78148f00 0x7efd781435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.915+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7efd780ce830 con 0x7efd78148f00 2024-05-27T03:34:12.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.915+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7efd780a9580 con 0x7efd640039b0 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.915+0000 7efd6ffff640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd7815db70 0x7efd781489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.915+0000 7efd6ffff640 1 -- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 msgr2=0x7efd7814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.916+0000 7efd6ffff640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd7814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.916+0000 7efd6ffff640 1 -- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd78148f00 msgr2=0x7efd781435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.916+0000 7efd6ffff640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd78148f00 0x7efd781435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.916+0000 7efd6ffff640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7815e5c0 con 0x7efd7815db70 2024-05-27T03:34:12.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.916+0000 7efd6ffff640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd7815db70 0x7efd781489c0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7efd6000c590 tx=0x7efd6000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:12.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.917+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60015070 con 0x7efd7815db70 2024-05-27T03:34:12.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.917+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd60011800 con 0x7efd7815db70 2024-05-27T03:34:12.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.917+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60011bf0 con 0x7efd7815db70 2024-05-27T03:34:12.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.917+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7815ee20 con 0x7efd7815db70 2024-05-27T03:34:12.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.917+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7815eb40 con 0x7efd7815db70 2024-05-27T03:34:12.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.919+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd78068b00 con 0x7efd7815db70 2024-05-27T03:34:12.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.921+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd60018070 con 0x7efd7815db70 2024-05-27T03:34:12.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.921+0000 7efd6f7fe640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7efd5806be80 0x7efd5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:12.922 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.922+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6623+0+0 (secure 0 0 0) 0x7efd6008ade0 con 0x7efd7815db70 2024-05-27T03:34:12.923 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.923+0000 7efd76d9d640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7efd5806be80 0x7efd5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:12.924 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.924+0000 7efd76d9d640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7efd5806be80 0x7efd5806e340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd781f3280 tx=0x7efd68009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:12.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:12.928+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd60058f20 con 0x7efd7815db70 2024-05-27T03:34:13.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.246+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd7813e460 con 0x7efd7815db70 2024-05-27T03:34:13.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.249+0000 7efd6f7fe640 1 -- 172.21.15.111:0/4063950118 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) ==== 100+0+11 (secure 0 0 0) 0x7efd6005d220 con 0x7efd7815db70 2024-05-27T03:34:13.254 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.255+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7efd5806be80 msgr2=0x7efd5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.255+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7efd5806be80 0x7efd5806e340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd781f3280 tx=0x7efd68009030 comp rx=0 tx=0).stop 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.255+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd7815db70 msgr2=0x7efd781489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.255+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd7815db70 0x7efd781489c0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7efd6000c590 tx=0x7efd6000ca50 comp rx=0 tx=0).stop 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 shutdown_connections 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7efd5806be80 0x7efd5806e340 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd781f3280 tx=0x7efd68009030 comp rx=0 tx=0).stop 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd78148f00 0x7efd781435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd7815db70 0x7efd781489c0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 --2- 172.21.15.111:0/4063950118 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7efd640039b0 0x7efd7814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 >> 172.21.15.111:0/4063950118 conn(0x7efd780723e0 msgr2=0x7efd780cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:13.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.256+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 shutdown_connections 2024-05-27T03:34:13.259 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.259+0000 7efd7c9e8640 1 -- 172.21.15.111:0/4063950118 wait complete. 2024-05-27T03:34:13.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T03:34:13.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.452+0000 7fd5355c7640 1 Processor -- start 2024-05-27T03:34:13.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.452+0000 7fd5355c7640 1 -- start start 2024-05-27T03:34:13.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.453+0000 7fd5355c7640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd5300735f0 0x7fd5300726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:13.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.453+0000 7fd5355c7640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd53006c980 0x7fd53006e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:13.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.453+0000 7fd5355c7640 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd530162a80 con 0x7fd53006e1a0 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.453+0000 7fd5355c7640 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 -- 0x7fd530162d50 con 0x7fd5300735f0 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.453+0000 7fd5355c7640 1 -- --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd530163030 con 0x7fd53006c980 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd5300735f0 0x7fd5300726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd534dc6640 1 --2- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd53006c980 0x7fd53006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd5300735f0 0x7fd5300726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.111:50616/0 (socket says 172.21.15.111:50616) 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 -- 172.21.15.111:0/3437844881 learned_addr learned my addr 172.21.15.111:0/3437844881 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52f7fe640 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd53006e1a0 0x7fd53006c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.111:50754/0 (socket says 172.21.15.111:50754) 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 -- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd53006c980 msgr2=0x7fd53006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 --2- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd53006c980 0x7fd53006e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 -- 172.21.15.111:0/3437844881 >> v1:172.21.15.153:6789/0 conn(0x7fd53006e1a0 legacy=0x7fd53006c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:13.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.454+0000 7fd52ffff640 1 -- 172.21.15.111:0/3437844881 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd530163310 con 0x7fd5300735f0 2024-05-27T03:34:13.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.456+0000 7fd52ffff640 1 --2- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5300735f0 0x7fd5300726c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd524009870 tx=0x7fd52402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972382d85c3b02a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:13.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.456+0000 7fd52effd640 1 -- 172.21.15.111:0/3437844881 <== mon.1 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037070 con 0x7fd5300735f0 2024-05-27T03:34:13.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.456+0000 7fd52effd640 1 -- 172.21.15.111:0/3437844881 <== mon.1 v2:172.21.15.153:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524032440 con 0x7fd5300735f0 2024-05-27T03:34:13.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd52effd640 1 -- 172.21.15.111:0/3437844881 <== mon.1 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524032810 con 0x7fd5300735f0 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd5355c7640 1 -- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5300735f0 msgr2=0x7fd5300726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd5355c7640 1 --2- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5300735f0 0x7fd5300726c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd524009870 tx=0x7fd52402a270 comp rx=0 tx=0).stop 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd5355c7640 1 -- 172.21.15.111:0/3437844881 shutdown_connections 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd5355c7640 1 --2- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd53006c980 0x7fd53006e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd5355c7640 1 --2- 172.21.15.111:0/3437844881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5300735f0 0x7fd5300726c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.457+0000 7fd5355c7640 1 -- 172.21.15.111:0/3437844881 >> 172.21.15.111:0/3437844881 conn(0x7fd5300cdf40 msgr2=0x7fd53006da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.458+0000 7fd5355c7640 1 -- 172.21.15.111:0/3437844881 shutdown_connections 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.458+0000 7fd5355c7640 1 -- 172.21.15.111:0/3437844881 wait complete. 2024-05-27T03:34:13.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.459+0000 7fd5355c7640 1 Processor -- start 2024-05-27T03:34:13.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.459+0000 7fd5355c7640 1 -- start start 2024-05-27T03:34:13.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.460+0000 7fd5355c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 0x7fd530149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd52ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 0x7fd530149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd52ffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 0x7fd530149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38186/0 (socket says 172.21.15.111:38186) 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd52ffff640 1 -- 172.21.15.111:0/877241586 learned_addr learned my addr 172.21.15.111:0/877241586 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53006e1a0 0x7fd5300b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5300735f0 0x7fd5300b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd530166270 con 0x7fd53006c980 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301638d0 con 0x7fd53006e1a0 2024-05-27T03:34:13.462 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.461+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] -- mon_getmap magic: 0 -- 0x7fd5301663f0 con 0x7fd5300735f0 2024-05-27T03:34:13.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.462+0000 7fd52ffff640 1 -- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5300735f0 msgr2=0x7fd5300b1960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T03:34:13.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.462+0000 7fd52ffff640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5300735f0 0x7fd5300b1960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.463 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.462+0000 7fd52ffff640 1 -- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53006e1a0 msgr2=0x7fd5300b1420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T03:34:13.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.462+0000 7fd52ffff640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53006e1a0 0x7fd5300b1420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.462+0000 7fd52ffff640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53014ed80 con 0x7fd53006c980 2024-05-27T03:34:13.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.463+0000 7fd52ffff640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 0x7fd530149030 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fd5240090b0 tx=0x7fd5240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:13.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.463+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524041020 con 0x7fd53006c980 2024-05-27T03:34:13.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.463+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53014dfc0 con 0x7fd53006c980 2024-05-27T03:34:13.466 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.466+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53014e580 con 0x7fd53006c980 2024-05-27T03:34:13.469 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.468+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5240309c0 con 0x7fd53006c980 2024-05-27T03:34:13.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.469+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037040 con 0x7fd53006c980 2024-05-27T03:34:13.470 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.469+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd524044020 con 0x7fd53006c980 2024-05-27T03:34:13.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.471+0000 7fd52cff9640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd51c06beb0 0x7fd51c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T03:34:13.471 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.471+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6623+0+0 (secure 0 0 0) 0x7fd5240af170 con 0x7fd53006c980 2024-05-27T03:34:13.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.473+0000 7fd52f7fe640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd51c06beb0 0x7fd51c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T03:34:13.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.473+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd510005290 con 0x7fd53006c980 2024-05-27T03:34:13.473 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.473+0000 7fd52f7fe640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd51c06beb0 0x7fd51c06e370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd530162050 tx=0x7fd5200079c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T03:34:13.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.482+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd5240392f0 con 0x7fd53006c980 2024-05-27T03:34:13.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.819+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd510005850 con 0x7fd53006c980 2024-05-27T03:34:13.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.819+0000 7fd52cff9640 1 -- 172.21.15.111:0/877241586 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) ==== 80+0+0 (secure 0 0 0) 0x7fd524079b80 con 0x7fd53006c980 2024-05-27T03:34:13.819 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2024-05-27T03:34:13.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.825+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd51c06beb0 msgr2=0x7fd51c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:13.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.825+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd51c06beb0 0x7fd51c06e370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd530162050 tx=0x7fd5200079c0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.825+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 msgr2=0x7fd530149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T03:34:13.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.825+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 0x7fd530149030 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fd5240090b0 tx=0x7fd5240046c0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 shutdown_connections 2024-05-27T03:34:13.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:6832/2917284088,v1:172.21.15.153:6833/2917284088] conn(0x7fd51c06beb0 0x7fd51c06e370 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fd5300735f0 0x7fd5300b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53006e1a0 0x7fd5300b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 --2- 172.21.15.111:0/877241586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd53006c980 0x7fd530149030 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T03:34:13.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 >> 172.21.15.111:0/877241586 conn(0x7fd5300cdf40 msgr2=0x7fd530163c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T03:34:13.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 shutdown_connections 2024-05-27T03:34:13.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-05-27T03:34:13.826+0000 7fd5355c7640 1 -- 172.21.15.111:0/877241586 wait complete. 2024-05-27T03:34:13.845 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T03:34:13.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T03:34:13.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-27T03:34:13.886 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T03:34:13.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T03:34:13.886 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T03:34:13.990 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-05-27T03:34:13.990 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T03:34:14.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18967 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 172.21.15.111:6789:/test /tmp/tmp.SabVhXOPketest ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=ef97d1dc-a8f7-4487-922a-ccc176f6927a,acl 0 0 2024-05-27T03:34:14.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-05-27T03:34:14.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T03:34:14.012 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-27T03:34:14.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T03:34:14.075 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-27T03:34:14.076 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T03:34:14.076 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T03:34:14.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056335,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-27T03:34:14.330 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T03:34:14.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T03:34:14.330 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-27T03:34:14.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T03:34:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-27T03:34:14.595 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-05-27T03:34:14.595 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T03:34:14.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056335,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=362 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-27T03:34:14.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2024-05-27T03:34:14.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T03:34:14.611 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-27T03:34:14.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T03:34:14.669 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-27T03:34:16.280 INFO:tasks.ceph.osd.0:Sent signal 15 2024-05-27T03:34:16.280 INFO:tasks.ceph.osd.1:Sent signal 15 2024-05-27T03:34:16.281 INFO:tasks.ceph.osd.2:Sent signal 15 2024-05-27T03:34:16.281 INFO:tasks.ceph.osd.3:Sent signal 15 2024-05-27T03:34:16.281 INFO:tasks.ceph.osd.4:Sent signal 15 2024-05-27T03:34:16.281 INFO:tasks.ceph.osd.6:Sent signal 15 2024-05-27T03:34:16.281 INFO:tasks.ceph.osd.7:Sent signal 15 2024-05-27T03:34:16.282 INFO:tasks.ceph.mds.a:Sent signal 15 2024-05-27T03:34:16.282 INFO:tasks.ceph.mds.c:Sent signal 15 2024-05-27T03:34:16.282 INFO:tasks.ceph.mds.b:Sent signal 15 2024-05-27T03:34:16.282 INFO:tasks.ceph.mds.d:Sent signal 15 2024-05-27T03:34:16.282 INFO:tasks.ceph.mon.a:Sent signal 15 2024-05-27T03:34:16.283 INFO:tasks.ceph.mon.b:Sent signal 15 2024-05-27T03:34:16.283 INFO:tasks.ceph.mon.c:Sent signal 15 2024-05-27T03:34:16.283 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-05-27T03:34:16.283 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-05-27T03:34:16.284 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f6764b08640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37706) UID: 0 2024-05-27T03:34:16.284 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f6764b08640 -1 osd.4 51 *** Got signal Terminated *** 2024-05-27T03:34:16.284 INFO:tasks.ceph.osd.4.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f6764b08640 -1 osd.4 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.284 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f2307fdb640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37710) UID: 0 2024-05-27T03:34:16.284 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f2307fdb640 -1 osd.6 51 *** Got signal Terminated *** 2024-05-27T03:34:16.285 INFO:tasks.ceph.osd.6.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f2307fdb640 -1 osd.6 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.285 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f62b9de8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37712) UID: 0 2024-05-27T03:34:16.285 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f62b9de8640 -1 osd.7 51 *** Got signal Terminated *** 2024-05-27T03:34:16.285 INFO:tasks.ceph.osd.7.smithi153.stderr:2024-05-27T03:34:16.282+0000 7f62b9de8640 -1 osd.7 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.285 INFO:tasks.ceph.mds.b.smithi153.stderr:2024-05-27T03:34:16.283+0000 7f36f8950640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 44821) UID: 0 2024-05-27T03:34:16.285 INFO:tasks.ceph.mds.b.smithi153.stderr:2024-05-27T03:34:16.283+0000 7f36f8950640 -1 mds.b *** got signal Terminated *** 2024-05-27T03:34:16.286 INFO:tasks.ceph.mds.d.smithi153.stderr:2024-05-27T03:34:16.284+0000 7f66a2c3c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 44822) UID: 0 2024-05-27T03:34:16.286 INFO:tasks.ceph.mds.d.smithi153.stderr:2024-05-27T03:34:16.284+0000 7f66a2c3c640 -1 mds.d *** got signal Terminated *** 2024-05-27T03:34:16.286 INFO:tasks.ceph.mon.b.smithi153.stderr:2024-05-27T03:34:16.284+0000 7f06b7c87640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 37124) UID: 0 2024-05-27T03:34:16.286 INFO:tasks.ceph.mon.b.smithi153.stderr:2024-05-27T03:34:16.284+0000 7f06b7c87640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-05-27T03:34:16.287 INFO:tasks.ceph.mon.c.smithi153.stderr:2024-05-27T03:34:16.284+0000 7ff35a580640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 37126) UID: 0 2024-05-27T03:34:16.287 INFO:tasks.ceph.mon.c.smithi153.stderr:2024-05-27T03:34:16.284+0000 7ff35a580640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-05-27T03:34:16.287 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T03:34:16.281+0000 7fb6912fe640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 37651) UID: 0 2024-05-27T03:34:16.287 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T03:34:16.281+0000 7fb6912fe640 -1 osd.0 51 *** Got signal Terminated *** 2024-05-27T03:34:16.287 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:34:16.281+0000 7fcaa45a7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 37658) UID: 0 2024-05-27T03:34:16.287 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:34:16.281+0000 7fcaa45a7640 -1 osd.1 51 *** Got signal Terminated *** 2024-05-27T03:34:16.288 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T03:34:16.282+0000 7f2e7fb39640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 37675) UID: 0 2024-05-27T03:34:16.288 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T03:34:16.282+0000 7f2e7fb39640 -1 osd.3 51 *** Got signal Terminated *** 2024-05-27T03:34:16.288 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T03:34:16.282+0000 7f3531e31640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 37663) UID: 0 2024-05-27T03:34:16.288 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T03:34:16.282+0000 7f3531e31640 -1 osd.2 51 *** Got signal Terminated *** 2024-05-27T03:34:16.288 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T03:34:16.282+0000 7f519a50e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 52522) UID: 0 2024-05-27T03:34:16.289 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T03:34:16.283+0000 7faf37c21640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 52523) UID: 0 2024-05-27T03:34:16.289 INFO:tasks.ceph.osd.1.smithi111.stderr:2024-05-27T03:34:16.289+0000 7fcaa45a7640 -1 osd.1 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.290 INFO:tasks.ceph.osd.3.smithi111.stderr:2024-05-27T03:34:16.289+0000 7f2e7fb39640 -1 osd.3 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.290 INFO:tasks.ceph.osd.0.smithi111.stderr:2024-05-27T03:34:16.289+0000 7fb6912fe640 -1 osd.0 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.290 INFO:tasks.ceph.osd.2.smithi111.stderr:2024-05-27T03:34:16.289+0000 7f3531e31640 -1 osd.2 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T03:34:16.291 INFO:tasks.ceph.mon.a.smithi111.stderr:2024-05-27T03:34:16.291+0000 7f45bb687640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 37376) UID: 0 2024-05-27T03:34:16.291 INFO:tasks.ceph.mon.a.smithi111.stderr:2024-05-27T03:34:16.291+0000 7f45bb687640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T03:34:16.295 INFO:tasks.ceph.mds.a.smithi111.stderr:2024-05-27T03:34:16.295+0000 7f519a50e640 -1 mds.a *** got signal Terminated *** 2024-05-27T03:34:16.295 INFO:tasks.ceph.mds.c.smithi111.stderr:2024-05-27T03:34:16.295+0000 7faf37c21640 -1 mds.c *** got signal Terminated *** 2024-05-27T03:34:16.490 INFO:tasks.ceph.mgr.y.smithi111.stderr:daemon-helper: command crashed with signal 15 2024-05-27T03:39:55.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:39:55.616 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:40:25.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:40:25.686 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:40:55.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:40:55.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:41:25.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:41:25.839 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:41:55.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:41:55.970 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:42:26.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:42:26.048 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:42:56.188 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:42:56.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:43:26.275 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:43:26.279 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:43:56.405 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:43:56.409 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:44:26.583 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:44:26.586 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:44:56.663 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:44:56.667 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:45:26.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:45:26.787 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:45:56.871 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:45:56.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:46:27.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:46:27.005 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:46:57.388 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:46:57.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:47:27.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:47:27.545 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:47:57.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:47:57.691 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:48:27.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:48:27.755 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:48:57.814 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:48:57.818 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:49:27.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:49:27.903 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:49:58.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:49:58.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:50:28.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:50:28.171 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:50:58.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:50:58.338 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:51:28.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:51:28.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:51:58.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:51:58.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:52:28.651 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:52:28.655 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:52:58.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:52:58.723 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:53:28.792 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:53:28.795 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:53:58.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:53:58.925 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:54:29.034 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:54:29.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:54:59.180 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:54:59.183 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:55:29.252 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:55:29.256 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:55:59.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:55:59.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:56:29.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:56:29.397 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:56:59.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:56:59.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:57:29.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:57:29.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:57:59.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:57:59.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:58:29.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:58:29.677 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:58:59.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:58:59.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:59:29.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:59:29.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:59:59.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T03:59:59.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:00:30.028 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:00:30.032 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:01:00.091 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:01:00.094 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:01:30.160 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:01:30.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:02:00.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:02:00.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:02:30.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:02:30.441 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:03:00.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:03:00.605 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:03:30.746 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:03:30.749 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:04:00.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:04:00.820 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:04:30.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:04:30.883 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:05:01.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:05:01.020 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:05:31.180 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:05:31.183 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:06:01.318 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:06:01.321 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:06:31.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:06:31.388 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:07:01.538 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:07:01.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:07:31.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:07:31.853 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:08:02.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:08:02.159 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:08:32.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:08:32.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:09:02.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:09:02.373 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:09:32.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:09:32.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:10:02.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:10:02.607 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:10:32.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:10:32.751 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:11:02.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:11:02.889 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:11:32.998 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:11:33.002 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:12:03.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:12:03.065 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:12:33.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:12:33.195 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:13:03.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:13:03.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:13:33.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:13:33.527 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:14:03.672 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:14:03.675 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:14:33.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:14:33.826 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:15:04.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:15:04.365 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:15:34.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:15:34.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:16:04.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:16:04.648 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:16:34.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:16:34.816 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:17:04.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:17:04.948 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:17:35.149 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:17:35.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:18:05.301 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:18:05.304 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:18:35.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:18:35.444 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:19:05.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:19:05.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:19:35.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:19:35.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:20:05.711 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:20:05.714 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:20:35.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:20:35.850 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:21:06.028 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:21:06.032 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:21:36.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:21:36.254 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:22:06.540 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:22:06.544 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:22:36.686 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:22:36.689 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:23:06.882 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:23:06.886 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:23:37.065 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:23:37.068 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:24:07.268 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:24:07.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:24:37.390 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:24:37.393 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:25:07.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:25:07.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:25:37.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:25:37.659 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:26:07.944 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:26:07.948 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:26:38.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:26:38.054 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:27:08.219 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:27:08.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:27:38.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:27:38.399 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:28:08.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:28:08.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:28:38.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:28:38.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:29:08.983 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:29:08.987 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:29:39.123 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:29:39.127 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:30:09.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:30:09.196 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:30:39.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:30:39.326 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:31:09.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:31:09.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:31:39.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:31:39.569 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:32:09.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:32:09.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:32:39.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:32:39.870 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:33:10.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:33:10.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:33:40.414 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:33:40.418 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:34:10.556 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:34:10.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:34:40.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:34:40.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:35:10.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:35:10.818 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:35:40.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:35:40.960 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:36:11.268 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:36:11.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:36:41.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:36:41.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:37:11.594 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:37:11.597 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:37:41.664 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:37:41.668 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:38:11.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:38:11.806 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:38:41.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:38:41.955 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:39:12.109 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:39:12.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:39:42.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:39:42.243 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:40:12.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:40:12.370 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:40:42.517 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:40:42.521 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:41:12.580 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:41:12.584 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:41:42.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:41:42.699 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:42:12.765 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:42:12.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:42:42.912 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:42:42.916 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:43:12.983 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:43:12.986 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:43:43.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:43:43.108 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:44:13.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:44:13.254 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:44:43.547 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:44:43.551 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:45:13.686 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:45:13.689 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:45:43.830 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:45:43.833 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:46:13.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:46:13.899 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:46:44.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:46:44.041 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:47:14.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:47:14.109 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:47:44.259 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:47:44.262 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:48:14.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:48:14.379 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:48:44.488 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:48:44.492 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:49:14.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:49:14.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:49:44.705 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:49:44.708 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:50:14.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:50:14.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:50:44.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:50:44.907 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:51:15.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:51:15.055 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:51:45.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:51:45.225 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:52:15.392 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:52:15.396 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:52:45.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:52:45.510 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:53:15.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:53:15.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:53:45.642 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:53:45.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:54:15.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:54:15.712 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:54:45.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:54:45.782 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:55:15.884 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:55:15.888 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:55:46.259 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:55:46.262 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:56:16.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:56:16.396 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:56:46.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:56:46.524 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:57:16.659 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:57:16.663 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:57:46.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:57:46.800 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:58:16.876 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:58:16.879 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:58:47.013 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:58:47.016 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:59:17.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:59:17.088 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:59:47.231 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T04:59:47.235 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:00:17.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:00:17.371 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:00:47.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:00:47.526 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:01:17.695 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:01:17.698 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:01:47.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:01:47.767 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:02:17.905 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:02:17.908 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:02:48.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:02:48.050 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:03:18.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:03:18.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:03:48.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:03:48.680 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:04:18.831 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:04:18.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:04:48.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:04:48.966 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:05:19.108 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:05:19.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:05:49.416 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:05:49.419 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:06:19.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:06:19.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:06:49.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:06:49.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:07:19.800 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:07:19.803 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:07:49.898 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:07:49.901 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:08:19.961 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:08:19.964 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:08:50.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:08:50.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:09:20.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:09:20.400 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:09:50.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:09:50.578 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:10:20.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:10:20.649 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:10:50.799 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:10:50.803 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:11:20.909 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:11:20.912 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:11:50.971 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:11:50.975 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:12:21.092 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:12:21.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:12:51.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:12:51.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:13:21.350 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:13:21.353 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:13:51.605 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:13:51.609 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:14:21.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:14:21.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:14:51.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:14:51.882 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:15:21.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:15:21.974 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:15:52.137 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:15:52.140 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:16:22.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:16:22.445 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:16:52.588 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:16:52.592 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:17:22.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:17:22.738 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:17:52.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:17:52.873 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:18:22.952 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:18:22.956 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:18:53.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:18:53.106 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:19:23.159 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:19:23.163 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:19:53.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:19:53.407 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:20:23.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:20:23.480 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:20:53.640 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:20:53.643 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:21:23.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:21:23.872 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:21:53.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:21:54.001 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:22:24.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:22:24.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:22:54.413 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:22:54.420 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:23:24.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:23:24.572 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:23:54.643 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:23:54.647 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:24:24.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:24:24.938 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:24:55.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:24:55.093 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:25:25.217 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:25:25.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:25:55.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:25:55.385 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:26:25.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:26:25.496 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:26:55.606 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:26:55.609 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:27:25.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:27:25.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:27:55.989 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:27:55.993 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:28:26.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:28:26.166 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:28:56.291 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:28:56.294 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:29:26.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:29:26.684 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:29:56.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:29:56.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:30:27.028 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:30:27.031 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:30:57.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:30:57.178 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:31:27.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:31:27.291 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:31:57.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:31:57.444 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:32:27.778 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:32:27.782 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:32:58.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:32:58.061 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:33:28.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:33:28.322 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:33:58.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:33:58.397 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:34:28.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:34:28.465 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:34:58.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:34:58.536 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:35:28.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:35:28.676 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:35:58.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:35:58.812 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:36:28.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:36:28.960 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:36:59.100 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:36:59.103 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:37:29.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:37:29.425 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:37:59.577 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:37:59.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:38:29.696 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:38:29.700 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:38:59.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:38:59.842 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:39:29.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:39:30.000 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:40:00.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:40:00.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:40:30.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:40:30.475 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:41:00.618 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:41:00.622 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:41:30.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:41:30.692 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:42:00.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:42:00.822 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:42:30.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:42:30.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:43:01.034 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:43:01.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:43:31.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:43:31.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:44:01.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:44:01.368 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:44:31.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:44:31.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:45:01.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:45:01.649 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:45:31.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:45:31.801 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:46:01.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:46:01.904 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:46:31.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:46:33.573 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:47:03.651 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:47:03.657 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:47:33.789 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:47:33.793 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:48:03.924 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:48:03.928 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:48:34.064 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:48:34.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:49:04.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:49:04.373 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:49:34.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:49:34.501 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:50:04.652 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:50:04.656 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:50:34.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:50:34.725 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:51:04.861 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:51:05.353 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:51:35.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:51:35.501 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:52:05.610 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:52:05.614 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:52:35.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:52:35.743 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:53:05.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:53:05.815 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:53:35.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:53:35.967 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:54:06.054 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:54:06.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:54:36.218 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:54:36.289 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:55:06.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:55:06.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:55:36.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:55:36.498 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:56:06.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:56:06.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:56:36.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:56:36.791 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:57:06.916 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:57:06.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:57:37.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:57:37.044 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:58:07.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:58:07.189 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:58:37.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:58:37.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:59:07.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:59:07.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:59:37.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T05:59:37.585 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:00:07.713 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:00:07.717 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:00:37.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:00:37.858 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:01:07.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:01:08.000 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:01:38.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:01:38.407 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:02:08.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:02:08.565 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:02:38.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:02:38.690 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:03:08.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:03:08.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:03:39.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:03:39.118 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:04:09.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:04:09.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:04:39.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:04:39.500 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:05:09.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:05:09.597 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:05:39.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:05:39.764 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:06:09.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:06:09.833 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:06:39.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:06:39.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:07:10.275 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:07:10.279 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:07:40.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:07:40.431 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:08:10.580 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:08:10.583 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:08:40.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:08:40.724 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:09:10.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:09:10.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:09:41.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:09:41.077 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:10:11.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:10:11.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:10:41.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:10:41.608 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:11:11.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:11:11.982 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:11:42.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:11:42.195 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:12:12.321 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:12:12.324 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:12:42.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:12:42.500 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:13:12.662 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:13:12.666 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:13:42.952 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:13:42.955 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:14:13.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:14:13.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:14:43.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:14:43.194 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:15:13.340 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:15:13.343 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:15:43.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:15:43.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:16:13.579 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:16:13.583 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:16:43.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:16:43.726 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:17:14.020 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:17:14.024 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:17:44.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:17:44.185 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:18:14.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:18:14.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:18:44.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:18:44.461 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:19:14.613 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:19:14.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:19:44.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:19:44.688 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:20:14.820 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:20:14.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:20:45.025 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:20:45.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:21:15.161 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:21:15.165 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:21:45.332 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:21:45.336 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:22:15.483 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:22:15.489 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:22:45.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:22:45.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:23:15.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:23:15.754 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:23:46.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:23:46.075 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:24:16.376 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:24:16.380 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:24:46.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:24:46.444 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:25:16.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:25:16.588 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:25:46.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:25:46.660 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:26:16.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:26:16.731 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:26:46.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:26:46.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:27:17.042 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:27:17.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:27:47.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:27:47.177 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:28:17.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:28:17.249 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:28:47.406 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:28:47.409 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:29:17.538 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:29:17.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:29:47.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:29:47.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:30:17.769 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:30:17.772 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:30:47.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:30:47.860 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:31:17.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:31:17.928 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:31:48.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:31:48.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:32:18.194 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:32:18.198 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:32:48.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:32:48.268 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:33:18.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:33:18.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:33:48.540 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:33:48.543 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:34:18.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:34:18.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:34:48.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:34:48.811 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:35:18.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:35:18.932 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:35:49.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:35:49.052 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:36:19.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:36:19.208 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:36:49.356 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:36:49.360 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:37:19.488 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:37:19.492 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:37:49.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:37:49.623 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:38:19.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:38:19.794 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:38:49.860 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:38:49.863 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:39:19.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:39:19.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:39:50.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:39:50.352 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:40:20.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:40:20.488 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:40:50.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:40:50.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:41:20.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:41:20.812 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:41:50.931 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:41:50.935 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:42:21.069 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:42:21.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:42:51.241 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:42:51.244 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:43:21.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:43:21.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:43:51.551 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:43:51.554 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:44:21.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:44:21.706 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:44:51.789 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:44:51.792 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:45:21.871 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:45:21.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:45:51.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:45:51.946 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:46:22.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:46:22.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:46:52.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:46:52.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:47:22.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:47:22.397 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:47:52.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:47:52.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:48:22.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:48:22.721 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:48:52.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:48:52.925 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:49:23.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:49:23.072 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:49:53.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:49:53.211 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:50:23.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:50:23.386 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:50:53.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:50:53.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:51:23.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:51:23.621 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:51:53.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:51:53.741 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:52:23.830 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:52:23.834 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:52:53.986 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:52:53.990 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:53:24.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:53:24.121 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:53:54.321 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:53:54.324 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:54:24.389 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:54:24.393 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:54:54.550 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:54:54.553 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:55:24.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:55:24.711 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:55:54.854 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:55:54.858 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:56:25.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:56:25.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:56:55.154 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:56:55.159 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:57:25.241 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:57:25.244 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:57:55.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:57:55.315 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:58:25.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:58:25.423 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:58:55.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:58:55.548 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:59:25.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:59:25.703 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:59:55.786 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T06:59:55.790 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:00:25.934 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:00:25.938 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:00:56.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:00:56.214 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:01:26.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:01:26.328 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:01:56.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:01:56.464 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:02:26.525 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:02:26.529 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:02:56.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:02:56.682 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:03:26.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:03:26.944 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:03:57.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:03:57.122 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:04:27.265 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:04:27.269 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:04:57.335 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:04:57.338 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:05:27.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:05:27.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:05:57.836 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:05:57.841 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:06:27.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:06:27.910 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:06:57.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:06:57.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:07:28.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:07:28.110 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:07:58.260 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:07:58.264 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:08:28.332 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:08:28.336 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:08:58.460 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:08:58.465 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:09:28.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:09:28.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:09:58.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:09:58.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:10:28.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:10:28.820 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:10:58.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:10:58.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:11:29.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:11:29.042 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:11:59.109 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:11:59.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:12:29.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:12:29.249 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:12:59.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:12:59.411 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:13:29.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:13:29.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:13:59.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:13:59.724 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:14:29.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:14:29.790 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:14:59.934 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:14:59.938 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:15:30.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:15:30.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:16:00.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:16:00.472 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:16:30.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:16:30.599 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:17:00.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:17:00.747 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:17:30.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:17:30.882 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:18:00.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:18:00.945 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:18:31.078 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:18:31.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:19:01.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:19:01.212 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:19:31.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:19:31.481 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:20:01.547 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:20:01.551 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:20:31.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:20:31.689 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:21:01.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:21:01.867 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:21:32.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:21:32.066 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:22:02.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:22:02.137 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:22:32.279 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:22:32.283 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:23:02.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:23:02.361 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:23:32.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:23:32.499 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:24:02.800 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:24:02.803 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:24:33.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:24:33.007 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:25:03.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:25:03.077 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:25:33.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:25:33.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:26:03.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:26:03.333 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:26:33.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:26:33.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:27:03.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:27:03.630 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:27:33.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:27:33.924 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:28:04.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:28:04.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:28:34.186 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:28:34.190 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:29:04.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:29:04.353 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:29:34.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:29:34.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:30:04.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:30:04.709 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:30:34.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:30:34.959 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:31:05.099 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:31:05.103 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:31:35.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:31:35.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:32:05.542 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:32:05.545 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:32:35.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:32:35.686 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:33:05.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:33:05.897 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:33:36.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:33:36.049 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:34:06.118 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:34:06.122 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:34:36.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:34:36.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:35:06.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:35:06.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:35:36.515 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:35:36.519 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:36:06.672 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:36:06.675 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:36:36.800 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:36:36.803 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:37:06.978 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:37:06.981 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:37:37.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:37:37.048 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:38:07.187 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:38:07.197 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:38:37.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:38:37.370 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:39:07.463 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:39:07.467 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:39:37.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:39:37.653 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:40:07.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:40:07.725 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:40:37.875 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:40:37.879 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:41:08.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:41:08.015 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:41:38.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:41:38.161 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:42:08.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:42:08.300 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:42:38.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:42:38.363 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:43:08.513 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:43:08.517 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:43:38.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:43:38.638 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:44:08.854 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:44:08.858 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:44:39.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:44:39.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:45:09.146 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:45:09.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:45:39.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:45:39.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:46:09.721 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:46:09.729 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:46:39.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:46:39.870 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:47:10.021 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:47:10.024 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:47:40.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:47:40.161 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:48:10.304 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:48:10.309 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:48:40.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:48:40.454 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:49:10.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:49:10.587 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:49:40.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:49:40.728 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:50:10.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:50:10.857 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:50:40.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:50:40.961 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:51:11.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:51:11.107 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:51:41.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:51:41.251 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:52:11.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:52:11.390 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:52:41.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:52:41.526 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:53:11.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:53:11.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:53:41.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:53:41.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:54:11.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:54:11.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:54:42.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:54:42.121 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:55:12.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:55:12.270 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:55:42.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:55:42.399 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:56:12.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:56:12.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:56:42.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:56:42.800 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:57:12.950 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:57:12.953 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:57:43.114 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:57:43.118 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:58:13.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:58:13.252 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:58:43.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:58:43.383 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:59:13.532 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:59:13.536 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:59:43.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T07:59:43.658 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:00:13.782 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:00:13.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:00:43.919 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:00:43.923 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:01:14.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:01:14.044 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:01:44.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:01:44.188 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:02:14.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:02:14.311 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:02:44.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:02:44.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:03:14.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:03:14.585 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:03:44.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:03:44.712 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:04:14.851 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:04:14.855 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:04:45.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:04:45.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:05:15.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:05:15.097 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:05:45.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:05:45.170 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:06:15.253 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:06:15.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:06:45.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:06:45.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:07:15.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:07:15.390 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:07:45.447 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:07:45.451 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:08:15.579 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:08:15.583 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:08:45.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:08:45.738 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:09:15.831 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:09:15.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:09:45.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:09:45.975 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:10:16.092 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:10:16.096 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:10:46.238 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:10:46.241 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:11:16.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:11:16.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:11:46.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:11:46.540 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:12:16.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:12:16.670 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:12:46.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:12:46.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:13:16.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:13:16.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:13:47.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:13:47.124 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:14:17.256 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:14:17.259 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:14:47.435 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:14:47.439 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:15:17.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:15:17.585 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:15:47.701 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:15:47.705 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:16:17.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:16:17.843 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:16:47.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:16:47.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:17:18.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:17:18.138 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:17:48.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:17:48.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:18:18.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:18:18.552 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:18:48.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:18:48.682 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:19:18.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:19:18.800 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:19:48.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:19:48.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:20:19.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:20:19.066 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:20:49.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:20:49.211 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:21:19.353 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:21:19.357 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:21:49.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:21:49.527 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:22:19.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:22:19.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:22:49.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:22:49.826 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:23:19.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:23:19.964 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:23:50.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:23:50.109 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:24:20.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:24:20.273 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:24:50.405 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:24:50.408 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:25:20.474 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:25:20.478 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:25:50.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:25:50.548 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:26:20.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:26:20.605 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:26:50.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:26:50.751 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:27:20.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:27:20.872 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:27:50.965 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:27:50.968 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:28:21.095 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:28:21.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:28:51.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:28:51.166 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:29:21.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:29:21.344 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:29:51.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:29:51.482 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:30:21.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:30:21.625 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:30:51.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:30:51.805 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:31:21.924 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:31:21.927 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:31:52.069 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:31:52.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:32:22.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:32:22.212 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:32:52.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:32:52.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:33:22.489 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:33:22.491 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:33:52.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:33:52.638 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:34:22.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:34:22.800 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:34:52.932 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:34:52.935 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:35:23.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:35:23.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:35:53.225 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:35:53.229 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:36:23.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:36:23.576 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:36:53.741 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:36:53.745 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:37:23.887 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:37:23.891 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:37:54.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:37:54.250 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:38:24.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:38:24.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:38:54.515 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:38:54.519 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:39:24.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:39:24.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:39:54.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:39:54.836 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:40:25.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:40:25.005 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:40:55.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:40:55.136 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:41:25.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:41:25.482 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:41:55.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:41:55.628 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:42:25.761 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:42:25.765 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:42:55.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:42:55.909 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:43:26.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:43:26.040 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:43:56.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:43:56.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:44:26.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:44:26.314 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:44:56.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:44:56.468 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:45:26.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:45:26.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:45:56.729 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:45:56.732 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:46:26.868 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:46:26.872 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:46:57.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:46:57.018 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:47:27.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:47:27.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:47:57.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:47:57.300 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:48:27.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:48:27.428 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:48:57.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:48:57.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:49:27.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:49:27.721 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:49:57.911 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:49:57.914 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:50:28.065 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:50:28.069 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:50:58.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:50:58.209 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:51:28.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:51:28.380 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:51:58.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:51:58.514 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:52:28.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:52:28.676 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:52:58.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:52:58.814 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:53:28.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:53:28.961 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:53:59.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:53:59.105 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:54:29.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:54:29.258 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:54:59.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:54:59.414 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:55:29.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:55:29.558 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:55:59.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:55:59.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:56:29.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:56:29.857 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:57:00.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:57:00.195 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:57:30.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:57:30.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:58:00.390 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:58:00.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:58:30.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:58:30.692 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:59:00.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:59:00.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:59:30.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T08:59:30.973 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:00:01.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:00:01.119 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:00:31.268 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:00:31.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:01:01.398 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:01:01.401 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:01:31.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:01:31.539 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:02:01.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:02:01.684 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:02:31.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:02:31.830 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:03:01.980 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:03:01.984 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:03:32.100 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:03:32.104 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:04:02.432 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:04:02.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:04:32.566 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:04:32.570 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:05:02.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:05:02.716 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:05:32.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:05:32.861 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:06:03.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:06:03.016 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:06:33.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:06:33.135 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:07:03.270 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:07:03.274 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:07:33.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:07:33.427 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:08:03.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:08:03.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:08:33.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:08:33.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:09:03.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:09:03.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:09:33.973 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:09:33.978 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:10:04.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:10:04.121 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:10:34.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:10:34.268 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:11:04.406 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:11:04.410 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:11:34.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:11:34.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:12:04.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:12:04.728 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:12:34.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:12:34.891 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:13:05.024 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:13:05.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:13:35.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:13:35.156 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:14:05.308 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:14:05.311 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:14:35.453 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:14:35.456 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:15:05.590 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:15:05.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:15:35.736 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:15:35.739 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:16:05.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:16:05.885 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:16:36.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:16:36.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:17:06.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:17:06.175 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:17:36.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:17:36.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:18:06.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:18:06.399 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:18:36.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:18:36.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:19:06.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:19:06.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:19:36.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:19:36.828 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:20:06.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:20:06.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:20:37.091 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:20:37.095 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:21:07.236 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:21:07.240 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:21:37.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:21:37.402 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:22:07.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:22:07.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:22:37.690 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:22:37.694 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:23:07.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:23:07.826 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:23:38.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:23:38.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:24:08.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:24:08.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:24:38.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:24:38.275 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:25:08.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:25:08.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:25:38.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:25:38.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:08.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:08.688 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:38.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:38.841 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:08.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:08.970 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:39.121 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:39.124 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:09.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:09.307 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:39.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:39.538 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:09.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:09.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:39.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:39.686 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:09.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:09.752 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:39.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:39.813 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:09.873 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:09.877 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:39.944 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:39.947 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:10.016 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:10.020 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:40.212 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:40.216 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:10.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:10.338 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:40.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:40.444 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:10.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:10.589 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:40.648 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:40.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:10.771 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:10.776 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:40.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:40.903 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:11.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:11.034 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:41.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:41.146 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:11.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:11.211 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:41.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:41.284 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:11.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:11.412 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:41.481 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:41.486 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:39:11.610 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:39:11.613 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:39:41.681 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:39:41.685 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:40:11.753 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:40:11.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:40:42.065 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:40:42.069 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:41:12.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:41:12.137 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:41:42.266 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:41:42.269 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:42:12.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:42:12.341 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:42:42.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:42:42.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:43:12.621 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:43:12.625 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:43:42.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:43:42.773 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:44:12.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:44:12.918 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:44:43.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:44:43.065 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:45:13.207 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:45:13.211 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:45:43.337 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:45:43.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:46:13.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:46:13.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:46:43.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:46:43.614 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:47:13.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:47:13.767 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:47:43.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:47:43.913 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:48:14.055 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:48:14.059 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:48:44.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:48:44.285 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:49:14.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:49:14.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:49:44.567 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:49:44.571 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:50:14.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:50:14.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:50:44.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:50:44.836 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:51:14.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:51:14.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:51:45.152 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:51:45.155 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:52:15.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:52:15.285 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:52:45.411 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:52:45.414 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:53:15.538 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:53:15.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:53:45.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:53:45.695 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:54:15.830 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:54:15.833 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:54:45.988 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:54:45.994 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:55:16.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:55:16.123 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:55:46.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:55:46.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:56:16.411 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:56:16.415 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:56:46.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:56:46.568 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:57:16.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:57:16.723 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:57:47.034 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:57:48.755 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:58:18.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:58:18.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:58:48.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:58:48.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:59:19.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:59:19.139 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:59:49.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:59:49.283 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:00:19.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:00:19.429 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:00:49.695 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:00:49.699 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:01:19.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:01:19.844 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:01:49.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:01:49.963 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:02:20.024 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:02:20.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:02:50.160 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:02:50.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:03:20.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:03:20.300 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:03:50.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:03:50.465 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:04:20.610 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:04:20.614 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:04:50.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:04:50.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:05:20.903 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:05:20.907 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:05:51.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:05:51.035 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:06:21.187 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:06:21.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:06:51.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:06:51.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:07:21.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:07:21.483 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:07:51.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:07:51.635 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:08:21.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:08:21.867 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:08:52.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:08:52.188 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:09:22.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:09:22.331 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:09:52.456 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:09:52.459 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:10:22.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:10:22.596 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:10:52.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:10:52.735 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:11:22.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:11:22.913 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:11:53.064 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:11:53.071 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:12:23.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:12:23.197 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:12:53.355 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:12:53.359 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:13:23.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:13:23.505 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:13:53.672 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:13:53.681 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:14:23.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:14:23.845 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:14:53.973 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:14:53.978 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:15:24.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:15:24.114 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:15:54.245 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:15:54.248 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:16:24.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:16:24.412 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:16:54.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:16:54.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:17:24.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:17:24.694 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:17:54.875 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:17:54.878 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:18:25.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:18:25.011 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:18:55.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:18:55.172 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:19:25.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:19:25.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:19:55.470 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:19:55.473 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:20:25.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:20:25.626 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:20:55.771 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:20:55.774 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:21:25.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:21:25.910 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:21:56.052 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:21:56.065 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:22:26.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:22:26.252 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:22:56.399 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:22:56.403 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:23:26.532 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:23:26.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:23:56.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:23:56.689 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:24:26.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:24:26.844 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:24:57.187 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:24:57.191 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:25:27.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:25:27.314 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:25:57.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:25:57.482 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:26:27.609 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:26:27.613 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:26:57.729 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:26:57.732 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:27:27.876 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:27:27.881 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:27:58.021 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:27:58.024 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:28:28.183 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:28:28.187 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:28:58.330 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:28:58.334 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:29:28.474 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:29:28.477 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:29:58.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:29:58.623 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:30:28.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:30:28.763 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:30:58.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:30:58.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:31:29.024 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:31:29.027 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:31:59.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:31:59.193 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:32:29.313 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:32:29.317 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:32:59.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:32:59.468 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:33:29.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:33:29.600 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:33:59.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:33:59.744 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:34:29.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:34:29.897 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:35:00.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:35:00.044 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:35:30.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:35:30.166 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:36:00.291 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:36:00.295 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:36:30.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:36:30.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:37:00.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:37:00.561 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:37:30.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:37:30.697 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:38:00.847 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:38:00.851 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:38:31.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:38:31.005 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:39:01.159 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:39:01.163 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:39:31.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:39:31.315 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:40:01.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:40:01.468 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:40:31.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:40:31.587 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:41:01.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:41:01.648 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:41:31.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:41:31.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:42:01.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:42:01.791 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:42:31.850 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:42:31.854 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:43:01.988 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:43:01.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:43:32.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:43:32.141 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:44:02.271 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:44:02.275 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:44:32.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:44:32.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:45:02.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:45:02.576 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:45:32.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:45:32.710 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:46:02.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:46:02.860 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:46:32.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:46:32.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:47:03.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:47:03.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:47:33.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:47:33.289 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:48:03.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:48:03.427 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:48:33.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:48:33.608 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:49:03.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:49:03.762 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:49:33.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:49:33.918 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:50:04.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:50:04.104 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:50:34.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:50:34.250 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:51:04.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:51:04.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:51:34.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:51:34.556 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:52:04.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:52:04.678 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:52:34.819 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:52:34.823 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:53:04.948 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:53:04.999 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:53:35.160 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:53:35.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:54:05.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:54:05.285 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:54:35.410 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:54:35.415 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:55:05.556 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:55:05.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:55:35.702 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:55:35.706 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:56:05.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:56:05.884 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:56:36.011 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:56:36.015 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:57:06.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:57:06.077 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:57:36.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:57:36.208 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:58:06.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:58:06.346 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:58:36.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:58:36.475 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:59:06.600 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:59:06.604 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:59:36.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T10:59:36.734 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:00:06.860 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:00:06.863 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:00:37.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:00:37.011 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:01:07.161 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:01:07.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:01:37.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:01:37.277 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:02:07.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:02:07.399 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:02:37.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:02:37.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:03:07.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:03:07.680 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:03:37.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:03:37.860 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:04:07.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:04:07.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:04:38.147 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:04:38.150 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:05:08.295 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:05:08.299 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:05:38.438 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:05:38.441 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:06:08.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:06:08.598 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:06:38.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:06:38.742 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:07:08.883 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:07:08.887 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:07:39.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:07:39.016 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:08:09.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:08:09.170 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:08:39.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:08:39.335 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:09:09.465 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:09:09.469 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:09:39.596 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:09:39.600 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:10:09.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:10:09.727 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:10:39.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:10:39.865 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:11:09.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:11:09.993 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:11:40.103 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:11:40.107 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:12:10.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:12:10.252 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:12:40.385 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:12:40.389 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:13:10.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:13:10.525 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:13:40.651 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:13:40.655 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:14:10.798 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:14:10.801 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:14:40.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:14:40.946 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:15:11.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:15:11.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:15:41.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:15:41.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:16:11.356 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:16:11.359 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:16:41.491 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:16:41.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:17:11.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:17:11.623 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:17:41.761 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:17:41.765 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:18:11.903 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:18:11.906 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:18:42.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:18:42.042 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:19:12.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:19:12.189 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:19:42.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:19:42.352 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:20:12.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:20:12.488 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:20:42.613 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:20:42.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:21:12.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:21:12.753 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:21:42.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:21:42.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:22:13.027 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:22:13.033 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:22:43.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:22:43.178 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:23:13.312 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:23:13.316 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:23:43.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:23:43.458 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:24:13.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:24:13.574 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:24:43.768 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:24:43.771 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:25:14.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:25:14.033 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:25:44.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:25:44.255 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:26:14.315 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:26:14.318 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:26:44.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:26:44.473 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:27:14.624 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:27:14.628 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:27:44.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:27:44.716 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:28:14.850 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:28:14.854 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:28:44.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:28:44.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:29:15.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:29:15.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:29:45.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:29:45.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:30:15.410 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:30:15.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:30:45.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:30:45.569 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:31:15.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:31:15.916 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:31:46.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:31:46.054 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:32:16.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:32:16.173 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:32:46.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:32:46.323 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:33:16.470 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:33:16.473 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:33:46.608 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:33:46.612 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:34:16.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:34:16.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:34:46.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:34:46.905 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:35:17.035 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:35:17.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:35:47.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:35:47.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:36:17.301 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:36:17.305 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:36:47.447 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:36:47.451 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:37:17.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:37:17.588 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:37:47.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:37:47.734 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:38:17.875 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:38:17.878 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:38:48.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:38:48.033 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:39:18.167 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:39:18.170 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:39:48.271 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:39:48.275 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:40:18.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:40:18.406 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:40:48.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:40:48.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:41:18.702 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:41:18.706 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:41:48.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:41:48.848 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:42:18.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:42:18.986 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:42:49.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:42:49.140 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:43:19.297 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:43:19.301 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:43:49.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:43:49.429 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:44:19.579 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:44:19.583 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:44:49.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:44:49.738 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:45:19.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:45:19.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:45:50.035 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:45:50.040 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:46:20.190 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:46:20.194 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:46:50.329 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:46:50.332 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:47:20.449 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:47:20.454 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:47:50.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:47:50.590 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:48:20.733 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:48:20.737 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:48:50.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:48:50.805 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:49:20.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:49:20.963 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:49:51.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:49:51.102 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:50:21.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:50:21.248 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:50:51.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:50:51.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:51:21.512 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:51:21.516 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:51:51.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:51:51.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:52:21.721 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:52:21.725 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:52:51.975 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:52:51.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:53:22.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:53:22.117 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:53:52.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:53:52.185 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:54:22.253 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:54:22.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:54:52.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:54:52.385 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:55:22.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:55:22.515 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:55:52.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:55:52.685 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:56:22.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:56:22.815 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:56:52.939 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:56:52.942 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:57:23.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:57:23.072 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:57:53.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:57:53.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:58:23.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:58:23.372 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:58:53.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:58:53.500 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:59:23.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:59:23.637 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:59:53.746 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T11:59:53.750 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:00:23.899 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:00:23.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:00:54.050 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:00:54.054 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:01:24.301 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:01:24.304 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:01:54.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:01:54.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:02:24.580 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:02:24.584 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:02:54.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:02:54.731 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:03:24.872 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:03:24.877 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:03:54.944 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:03:54.947 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:04:25.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:04:25.018 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:04:55.086 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:04:55.089 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:05:25.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:05:25.160 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:05:55.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:05:55.306 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:06:25.432 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:06:25.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:06:55.562 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:06:55.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:07:25.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:07:25.695 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:07:55.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:07:55.858 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:08:25.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:08:25.996 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:08:56.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:08:56.135 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:09:26.243 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:09:26.247 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:09:56.373 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:09:56.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:10:26.486 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:10:26.491 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:10:56.650 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:10:56.653 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:11:26.713 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:11:26.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:11:56.848 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:11:56.851 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:12:26.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:12:26.973 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:12:57.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:12:57.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:13:27.227 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:13:27.231 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:13:57.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:13:57.342 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:14:27.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:14:27.480 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:14:57.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:14:57.592 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:15:27.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:15:27.738 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:15:57.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:15:57.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:16:28.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:16:28.012 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:16:58.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:16:58.175 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:17:28.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:17:28.311 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:17:58.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:17:58.440 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:18:28.573 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:18:28.577 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:18:58.711 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:18:58.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:19:28.848 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:19:28.852 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:19:59.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:19:59.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:20:29.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:20:29.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:20:59.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:20:59.247 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:21:29.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:21:29.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:21:59.527 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:21:59.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:22:29.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:22:29.684 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:22:59.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:22:59.822 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:23:29.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:23:29.967 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:24:00.109 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:24:00.114 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:24:30.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:24:30.250 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:25:00.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:25:00.370 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:25:30.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:25:30.499 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:26:00.642 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:26:00.646 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:26:30.762 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:26:30.766 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:27:00.908 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:27:00.911 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:27:31.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:27:31.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:28:01.199 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:28:01.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:28:31.386 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:28:31.389 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:29:01.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:29:01.671 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T12:29:10.659 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-05-27T12:29:10.659 DEBUG:teuthology.task.console_log:Killing console logger for smithi111 2024-05-27T12:29:10.660 DEBUG:teuthology.task.console_log:Killing console logger for smithi153 2024-05-27T12:29:10.660 DEBUG:teuthology.exit:Finished running handlers